Grouper Call of Nov. 22, 2023
Attending
- Chris Hyzer, Penn, Chair
- Shilen Patel, Duke
- Chad Redmond, Unicon
- Jim Beard , Unicon
- Vivek Sachdiva, independent
- Bert Bee Lindgren, GA Tech
- Gail Lift, Univ of Michigan
- Daniel Fisher, Virginia Tech
- Chris Hubing, Internet2
- Emily Eisbruch, Independent, scribe
Administrivia
- Internet2 Intellectual Property Policy
- Review AIs Grouper Project Action Items (Google Doc)
- Agenda Bash
New Action Items from this call
- Vivek - add the version to which this applies to on the Migrate from json-lib to jackson wiki documentation=
- Chris Hyzer and Shilen - discuss next steps on instrumentation data
Current Work
Vivek
- Worked on Migrate from json-lib to jackson
- Converting from JSON-lib to Jackson and back
- Goal is move to Jackson
- During testing we found some issues
- It is suggested to have option for users to still use Json-Lib
- Now there’s a property you can set true or false
- By default, use Jackson, Jackson is faster
- The two approaches are similar, but Jackson will go through the object model and figure out what it's going to do. Then the work is fast.
- JSON-lib is dynamic and performance issues
- Chad has scripts using json-lib, would prefer to use Jackson
- AI , Vivek will add the version this applies to on the Migrate from json-lib to jackson documentation
- LDAP list of groups from attributes and list in loader - that work is done
- Default setting is put groups next to where loader job is configured
- And you specify a folder that exists, it will use that
- For a new loader job with an absolute path, you can put it anywhere in the registry
- In some future version, change the default perhaps
- Best of both worlds, can use the default and you can also put things where you want
- Vivek worked on GRP-5082 ldap loader LDAP_GROUPS_FROM_ATTRIBUTES should allow specifying parent stem
- Chris showed the Group edit screen
- If you have admin you can edit attestation on a group
- Can do customization
- By default, environment is not shown
Shilen
- Ldap
- Found several things to adjust; these are fixed
- Doing queries against API, there were errors
- Needed changes on normalization
- Old code was ignoring certain errors
- Code was not handling moving objects in provisioners
- Pools weren’t being pulled properly, causing many error messages
- Question: Do we need unbound ID unit tests?
- Shilen : don’t know, will take a look, see pull requests Daniel did
- Shilen also made changes to data provider in V5
- Now it compares data stamps
- It makes sure incremental stuff is not older than last pull for that data provider
- Next for Shilen: integrating GSH templates for data provider
- Dan: this merge went well
- Shilen ran tests, there were just a few issues found and fixed
- Unbound ID for test scope is needed
- Added a dependency to run unit tests around ldaptive session class
- Can get rid of that dependency if needed
- Ldaptive in v5 is snapshot
- That will get fixed with next release
- Helps in fixing bug
- Chris hopes to release a new Grouper v5
- Dependency on snapshot ….
- Dan will notify Chris Hyzer on Slack when ready for next steps, in addition to doing a Pull Request
At Penn, wanted to move to more Duo native
- Chris - there are 2 step users
- Used for policies
- Sync from Duo to database table to see if you are enrolled
- Now Salepoint is managing that, it is provisioning the enrolled in 2 step role
- Question of what needs to be in a transaction
- Permission groups
- Issue of adding a person to a group
- User expectation issue
- User hits save but then must get feedback, so there are not wrong assumptions
- Close the loop so user can verify, Bob was added to group and it was fully effective as of this time
Chris
- Chris worked on Group edit screen
Memory issue
- U. Michigan found memory change 4.5 to 4.8 . See JIRA 4950, but different conditions.
- Michigan can provide logs
- Running out of Java memory
- Connection pool errors
- When stagger the provisioning runs
- Could be a setting
- Chris suggests this may be fixed in Grouper 4.9
- Suggestion: look at connection pooling
- But that might not explain the memory issue
Chad
- Unicon has Authentication plug in
- JJ has a web app for unit testing
JJ has raised issues:
- Folder/Directory names
- Non standard folder names for sources
- Makes it challenging for grouper beginners
- Structure was decided many years ago
- Hope to be as consistent as possible
- Consistency with Shib model?
- Issue with tracking java file
- Chad created a branch with greater consistency
- Nice to have authentication in the UI built-in
- Gail: did not notice the Grouper source code was not Maven standard
- We will change build scripts
- Biggest challenge will be different file locations and tracking file history
- Question: Better to do this change of folder names on a major Grouper version change?
- Chris Hyzer: not really, originally that was the plan, but when we compare with another branch, we want comparisons to work.
- Container
- Issue is that it is hard to built container since it builds from an installer
- Suggestion from Chad: get rid of installer and use script
- Installer is used only for build script
- Installer has been taken out of training
- Scripts in container, if we need to do something, we have it
- Chris does not like using the Docker commands, too many layers
- Problem: if you want to change script, you must build Grouper to Maven
- And then you must release it
- Or change Docker file to use local Grouper, which is faking it
- Decision: Chris Hyzer is OK with this plan to move away from the Grouper Installer in favor of script
- Grouper Client
- client is intended to be standalone Jar, can be run on command line without dependencies
- Like that model, without dependencies
- Maven packaging can do that
- We have extra stuff
- Jackson, refactor
- Tried shade plug in from Maven
- Next Steps
- Chad will handle client first
- then build script
- then change directory names
- Good to move authentication into Grouper -Misc
- Get JAR published in maven
- Should be in plug-ins directory
- Documentation - we need a wiki about this
- Properties set in Grouper config files should be in the base
- Grouper web app….
- Snapshot issue
- Docker would get from local maven
- Built for releases, they are on Maven Central
- Talk to Maven to get snapshots, or Bert can tell how it’s done
Grouper Training
- Grouper Training is coming up Dec 12-15, 2023, there will minor changes
- For the Grouper training in January 2024, there will be more substantial changes
- Hoping for some on-demand training options
- from last Grouper call: Mural board, please submit your ideas
- https://app.mural.co/t/internet28867/m/internet28867/1699451419046/3a8368531a966d626921d4a81555fcf58e609116?sender=u1bf24de66f99afca0a366077
Issue Roundup
Jiras in past two weeks
GRP-5143
config metadata duplicate regexp causes "Same config key or regex is in multiple files"
GRP-5142
change from json-lib to jackson for json mashalling
GRP-5141
upgrade and refactor legacy google apps provisioner for security issue
GRP-5140
upgrade jose4j jar in legacy box provisioner
GRP-5139
upgrade oauth2 nimbus jar for security issue
GRP-5138
upgrade mysql driver for security issue
GRP-5137
make sure old fileupload jar is not in container
GRP-5136
Remove forked classes in ext and extMore
GRP-5135
Rewrite container installer as a script
GRP-5134
Reorganize source directories to be more standard
GRP-5133
Loader Subjob Query: Missing index
GRP-5132
NPE in GrouperLoaderDb.retrieveDataSourceFromC3P0
GRP-5131
dont allow config ids with private or pass or other things that autoencrypt...
GRP-5130
remove json-lib from zoom (migrate to jackson)
GRP-5129
add option (default true) that if zoom users are reactivated, they will also be licensed
GRP-5128
add basic auth to scim for grouper
GRP-5127
grouperProvisioningGroup fields not available during delete, gives NPE
GRP-5126
If a loader display name has the wrong number of colons, it uses the parent extension twice
GRP-5125
LDAP group provisioner getting wrong counts of members
GRP-5124
GrouperProvisioningMatchingIdIndex.mergeInNewTargetGroups error
GRP-5123
pull request for adding content type to grouper client
GRP-5122
upgrade org.json to 20231013 for security issue
GRP-5121
upgrade amqp-client to 5.20.0 for security vulnerability
GRP-5120
add visibility easy metadata option for azure
GRP-5119
entityResolver ldapMappingEntityAttribute only allows subjectId or subjectIdentifier0
GRP-5118
allow provisionable for small groups
GRP-5117
simplified UI for GSH templates (Security and "return to full UI" feature request)
GRP-5116
start the v5 container as tomcat
GRP-5115
Report gsh scripts can't be >4000 characters
GRP-5114
apache config needs to log client ip address if using reverse proxy with remote header
GRP-5113
group_id_index issue with midpoint provisioner
GRP-5112
md_grouper_allowProvisionableRegexOverride use causes errors when importing from gsh script
GRP-5111
fix newlines in email rules (plain text)
GRP-5110
loader list doesnt show disab
Grouper Emails in past two weeks
- [grouper-users] The keystore update GRP-4932 disable the documented Trusted roots in anchors folder, Krenn, Martin, 11/21/2023
(Emily emailed Martin Krenn Nov 22, 2023, suggesting he ask this on InCommon Grouper Slack, as Grouper-Users is not monitored by the Grouper Dev Team)
Grouper wiki updates in past two weeks
- v4 Release Notes
- SLAC Grouper container in Amazon AWS ECS
- SLAC National Accelerator Laboratory at Stanford University
- Elastic Container Service
- v4 Upgrade instructions from v4
- Grouper documentation pages to update
- Find a folder or a group by navigation
- Getting Started with Grouper
- Install the Grouper v2.5 container maturity level -1 quick start v2.6.5+
- Grouper ABAC Crashplan deprovisioning example
- Grouper custom template via GSH - V2
- Grouper configuration in UI readwrite mode
- Assign someone to be able to manage a group
- Migrate from json-lib to jackson
- Grouper Book - Web service
- Exposing Groups Through Shibboleth
- Create a composite group
Next Grouper Call: Wed., Dec. 6, 2023