Grouper Call of Oct. 26, 2022
Attending
- Chris Hyzer, Penn, Chair
- Shilen Patel, Duke
- Vivek Sachdiva, independent
- Chad Redmon, UNC
- Carey Black, Purdue
- Drew Aschenbrener, Internet2
- Chris Hubing, Internet2
- Emily Eisbruch, Internet2
Next Action Items from this call
- Chris - create wiki about provisioning error codes and explain what they are
- Chad - update wiki for new SCIM provisioner
- Chad - add flag to enable SCIM provisioner and default it to TRUE
Administrivia
- Internet2 Intellectual Property Policy
- Review AIs Grouper Project Action Items (Google Doc)
- Agenda Bash
Current Projects
Vivek
- Azure provisioning
- Throttle issue
- Batch, look at return code, see the throttle seconds, 150 seconds often, max number of seconds returned from items in the batch, it will sleep that amount of time plus 5 seconds buffer, then redo the ones that got the 429. If it works in threads it still works.
- Should address the 2 issues at U-MICH: throttling and missing members
- Question : how does a local deployer understand the start and stop of provisioning? Will logs show that throttling was happening?
- What do you do to validate the value you get back?
- Count throttles, count seconds, have a max
- Vivek will work on Logging
- Crud default - only for track groups
Shilen
- Making the provisioner work better with unresolvable subjects
- Gets marked in Grouper Members Table as unresolvable
- After 30 days it will get marked as deleted
- But if the subject is marked as deleted, it should not get pulled into what the provisioner is looking at
- If subject is unresolvable but not deleted, by default it is pulled in to provision, but you should have option to treat the subject as if deleted for purpose of provisioning
- Code does validation
- If entity is not provisionable , but is in the target, then even if it is going to delete the entity from target it will end up with an error
- Shilen added tweak to skip validation
- Shilen will look at grouper sync membership table
- Entity updates issue can get complex
Chris
- Met with community members from Harvard, Illinois State
- walked them through the provisioner
- It will be important to provide support to the community
- Getting better ideas for documentation
- Grouper 2.6.16 is working well
- Grouper 2.6.17 will have new Azure work, and an improvements on selecting
- Have test case for Azure, should have test case for all the provisioners
- Hope to release 2.6.17 within a few days
- Chad notes that he has been supporting community members provisioning to Azure with Grouper v2.6.16 and there have been questions.
- AI Chris create wiki about provisioning error codes and explain what they are
- Carey: have a group in Grouper for people who did not make it in provisioning?
- Chris will think about this. Maybe have provisioning screen easy view. Now you can go to activity log in UI. But will try to make it easier, on membership screen
- Some would expect to see this on the provisioning screen
- In Grouper 2.7
- See Roadmap
- we want a pared down container
- To improve performance
- Start using new DDL strategy
- New integer primary key
- Dictionary table
- Go thru existing tables and add new columns
- Migration to Grouper 3.0 with changing tables
- Changing from Guids (40 chars) to integer IDs (8 bytes or 22 bytes) will be helpful.
- Will have a storage, network and memory benefit
- Less indexes
- Need on groups, members, maybe on stems
- Then build new system with ideal more database design
- Issues:
- Concern for how difficult it might be to do the upgrade for a current deployer
- Custom queries may need to be upgraded
- Loader jobs that depend on Grouper tables will need to be upgraded
Chad
- SCIM provisioning, first pass is done, it’s in web services
- Get off of TomEE
- AI Chad - update wiki for new SCIM provisioner
- AI Chad - add flag to enable SCIM provisioner and default it to TRUE
- Working w customer has Grouper with AWS / SQL
- Looked at that with new provisioner
- Lookup
- Penn uses table in AWS for subject source
- Chris will share configurations
- Use Daemon
- Do a SQL sync job, full and incremental
- Put a trigger and change log table
- Then can do incremental updates and full sync and it works quickly
- Once we have data fields the remote reaching out to get users will go away
- There are some issues with the Defaults, MAX RESULTS said 100 but was unlimited
- Chad tried to set it to small value, but doing batch lookups for a loader, it batches
- MAX RESULTS must be at least 180
- It’s not documented well
- Do we need the new default?
- There is already a jira
- Postgress max connections, defaulted to 100
- There are a lot of brief pages on postgress and none mention the max connections
- Made a change on the specsheet
- Chad and Chris Hubing are looking at Jenkins Java
- Java 11 versus Java 17
- Things more likely to work with Java 11 since it’s closer to where we are
- We should try Java 17 and find out if there is any blocker
- Better to use the latest version if possible
Issue Roundup
Jiras in past two weeks
GRP-4445
SQL subject source should show form field for maxPageSize
GRP-4444
show adds and updates and deletes if there is a long attribute list in provisioning logs
GRP-4443
run harvard group of names add a large group, takes 20 minutes
GRP-4442
allow nulls in jexl scripts
GRP-4440
provisioning translate DNs even if they are not cached
GRP-4439
Subject source adapters don't limit query to default limit when max results is blank
GRP-4438
azure error on memberships (null pointer)
GRP-4437
syncing large group to azure misses some members
GRP-4436
WS SCIM 2 allow to enable/disable via configuration property
GRP-4435
WS SCIM 2 to implement PATCH method
GRP-4434
WS SCIM 2 to implement Bulk updates
GRP-4433
WS SCIM 2 to implement /Schemas endpoint
GRP-4432
Rewrite SCIM 2 service to remove dependence on J2EE and TomEE
GRP-4431
grouper junit should shut down after test after a delay, and cancal shutdown on startup
GRP-4430
upgrade commons text
GRP-4429
MembershipSave.save() doesn't return a membership on insert
GRP-4428
For SQL subject source, add option for search column to wildcard just the suffix
GRP-4427
adjust mailNickname documentation to note the max length
GRP-4426
do not symlink /run/secrets
GRP-4425
provisioning edit provisionable if set provision to No, then it doesnt save (when previously provisionable)
GRP-4424
translate objects with default values
GRP-4423
provisioner exception
GRP-4422
allow other system to remove memberships in provisioning
GRP-4421
change dockerfile to get java from rpm and not direct install
Grouper Emails in recent week
- [grouper-users] edu.internet2.middleware.grouperClient.ws.GcWebServiceError: Bad response from web service: resultCode: PROBLEM_GETTING_MEMBERS, Sahull, 09/14/2022
- [grouper-users] Azure Provisioner null pointer exception error, Sahull, 09/15/2022
- [grouper-users] fun with DDL upgrade, Jeff McCullough, 09/21/2022
- [grouper-users] using two instances of PSPNG, Ben Beecher, 09/21/202
Thanks Chad for responding to this
[grouper-users] Peer help required to smuggle uid shib var to grouper-ui, Francesco Malvezzi, 10/05/2022
- Re: [grouper-users] Peer help required to smuggle uid shib var to grouper-ui, Chad Redman, 10/05/2022
- Re: [grouper-users] Peer help required to smuggle uid shib var to grouper-ui, Francesco Malvezzi
Grouper wiki updates in past two weeks