Grouper Call of Oct. 26, 2022

Attending 

• Chris Hyzer, Penn, Chair
• Shilen Patel, Duke
• Vivek Sachdiva, independent  
• Chad Redmon, UNC
• Carey Black, Purdue
•  Drew Aschenbrener, Internet2
• Chris Hubing, Internet2
• Emily Eisbruch, Internet2

Next Action Items from this call

• Chris - create wiki about provisioning error codes and explain what they are
  
• Chad - update wiki for new SCIM provisioner
  
• Chad - add flag to enable SCIM provisioner and default it to TRUE
  

 Administrivia

•  Internet2 Intellectual Property Policy
  
• Review AIs  Grouper Project Action Items (Google Doc)
  
•  Agenda Bash
  

Current Projects

Vivek

• Azure provisioning
  
• Throttle issue
  
• Batch, look at return code, see the throttle seconds, 150 seconds often, max number of seconds returned from items in the batch, it will sleep that amount of time plus 5 seconds buffer, then redo the ones that got the 429.  If it works in threads it still works.
  
• Should address the 2 issues at U-MICH: throttling and missing members 
  
• Question : how does a local deployer understand the start and stop of provisioning? Will logs show that throttling was happening? 
  
• What do you do to validate the value you get back?
  
• Count throttles, count seconds, have a max
  
• Vivek will work on Logging
  
  
  
• Crud default - only for track groups
  

Shilen

• Making the provisioner work better with unresolvable subjects
  
• Gets marked in  Grouper Members Table as unresolvable
  
•   After 30 days it will get marked as deleted
  
• But if the subject is marked as deleted, it should not get pulled into what the provisioner is looking at
  
• If subject is unresolvable but not deleted, by default it is pulled in to provision, but  you should have option to treat the subject as if deleted for purpose of provisioning
  
• Code does validation
  
• If entity is not provisionable , but is in the target, then even if it is going to delete the entity from target it will end up with an error 
  
• Shilen added tweak to skip validation 
  
• Shilen will look at grouper sync membership table
  
• Entity updates issue can get complex
  
  

Chris

• Met with community members from Harvard, Illinois State
  
• walked them through the provisioner
  
• It will be important to provide support to the community
  
• Getting better ideas for documentation 
  
• Grouper 2.6.16 is working well
  
• Grouper 2.6.17 will have new Azure work, and an improvements on selecting
  
• Have test case for Azure, should have test case for all the provisioners
  
• Hope to release 2.6.17 within a few days
  
• Chad notes that he has been supporting community members provisioning to Azure with Grouper v2.6.16 and there have been questions. 
  
• AI Chris create wiki about provisioning error codes and explain what they are
  
• Carey: have a group in Grouper for people who did not make it in provisioning? 
  
• Chris will think about this. Maybe have provisioning screen easy view.  Now you can go to activity log in UI. But will try to make it easier, on membership screen
  
• Some would expect to see this on the provisioning screen
  

• In Grouper 2.7 
  
• See Roadmap
• we want a pared down container
  
• To improve performance 
  
• Start using new DDL strategy 
  
• New integer primary key
  
• Dictionary table
  
• Go thru existing tables and add new columns
  
• Migration to Grouper 3.0 with changing tables
  
•   Changing from Guids (40 chars) to integer IDs (8  bytes or 22  bytes) will be helpful.
  
• Will have a storage, network and memory benefit
  
• Less indexes
  
• Need on groups, members, maybe on stems
  
• Then build new system with ideal more database design
  
• Issues:
  
•   Concern for how difficult it might be to do the upgrade for a current deployer
  
•   Custom queries may need to be upgraded
  
•   Loader jobs that depend on Grouper tables will need to be upgraded
  

Chad

• SCIM provisioning, first pass is done, it’s in web services
  
• Get off of TomEE
  
• AI Chad - update wiki for new SCIM provisioner
  
• AI Chad - add flag to enable SCIM provisioner and default it to TRUE
  
  
  
  
• Working w customer has Grouper with AWS / SQL 
  
• Looked at that with new provisioner
  
• Lookup
  
• Penn uses table in AWS for subject source
  
• Chris will share configurations
  
• Use Daemon
  
• Do a SQL sync job, full and incremental
  
• Put a trigger and change log table
  
• Then can do incremental updates and full sync and it works quickly
  
• Once we have data fields the remote reaching out to get users will go away
  
• There are some issues with the Defaults, MAX RESULTS said 100 but was unlimited
  
• Chad tried to set it to small value, but doing batch lookups for a loader, it batches
  
• MAX RESULTS must be at least 180
  
• It’s not documented well
  
• Do we need the new default?
  
• There is already a jira
  
  
  
• Postgress max connections, defaulted to 100
  
• There are a lot of brief pages on postgress and none mention the max connections
  
• Made a change on the specsheet
  
  
• Chad and Chris Hubing are looking at Jenkins Java
  
  
•  Java 11 versus Java 17
  
• Things more likely to work with Java 11 since it’s closer to where we are
  
• We should try Java 17 and find out if there is any blocker
  
• Better to use the latest version if possible
  

Issue Roundup 

Jiras in past two weeks

GRP-4445
SQL subject source should show form field for maxPageSize

• 
  GRP-4444
  show adds and updates and deletes if there is a long attribute list in provisioning logs
  
  
  
  
• 
  GRP-4443
  run harvard group of names add a large group, takes 20 minutes
  
  
  
  
• 
  GRP-4442
  allow nulls in jexl scripts
  
  
  
  
• 
  GRP-4440
  provisioning translate DNs even if they are not cached
  
  
  
  
• 
  GRP-4439
  Subject source adapters don't limit query to default limit when max results is blank
  
  
  
  
• 
  GRP-4438
  azure error on memberships (null pointer)
  
  
  
  
• 
  GRP-4437
  syncing large group to azure misses some members
  
  
  
  
• 
  GRP-4436
  WS SCIM 2 allow to enable/disable via configuration property
  
  
  
  
• 
  GRP-4435
  WS SCIM 2 to implement PATCH method
  
  
  
  
• 
  GRP-4434
  WS SCIM 2 to implement Bulk updates
  
  
  
  
• 
  GRP-4433
  WS SCIM 2 to implement /Schemas endpoint
  
  
  
  
• 
  GRP-4432
  Rewrite SCIM 2 service to remove dependence on J2EE and TomEE
  
  
  
  
• 
  GRP-4431
  grouper junit should shut down after test after a delay, and cancal shutdown on startup
  
  
  
  
• 
  GRP-4430
  upgrade commons text
  
  
  
  
• 
  GRP-4429
  MembershipSave.save() doesn't return a membership on insert
  
  
  
  
• 
  GRP-4428
  For SQL subject source, add option for search column to wildcard just the suffix
  
  
  
  
• 
  GRP-4427
  adjust mailNickname documentation to note the max length
  
  
  
  
• 
  GRP-4426
  do not symlink /run/secrets
  
  
  
  
• 
  GRP-4425
  provisioning edit provisionable if set provision to No, then it doesnt save (when previously provisionable)
  
  
  
  
• 
  GRP-4424
  translate objects with default values
  
  
  
  
• 
  GRP-4423
  provisioner exception
  
  
  
  
• 
  GRP-4422
  allow other system to remove memberships in provisioning
  
  
  
  
• 
  GRP-4421
  change dockerfile to get java from rpm and not direct install
  

Grouper Emails in recent week

• [grouper-users] edu.internet2.middleware.grouperClient.ws.GcWebServiceError: Bad response from web service: resultCode: PROBLEM_GETTING_MEMBERS, Sahull, 09/14/2022
  
• [grouper-users] Azure Provisioner null pointer exception error, Sahull, 09/15/2022
  
• [grouper-users] fun with DDL upgrade, Jeff McCullough, 09/21/2022
  
• [grouper-users] using two instances of PSPNG, Ben Beecher, 09/21/202
  

Thanks Chad for responding to this

[grouper-users] Peer help required to smuggle uid shib var to grouper-ui, Francesco Malvezzi, 10/05/2022

• Re: [grouper-users] Peer help required to smuggle uid shib var to grouper-ui, Chad Redman, 10/05/2022
  
• Re: [grouper-users] Peer help required to smuggle uid shib var to grouper-ui, Francesco Malvezzi
  

Grouper wiki updates in past two weeks

• Penn membership requirements
  
• How to Setup an ultra-lite GSH Grouper Development Environment for Grouper v2.6
  
• v2.5 Release Notes
  
• Grouper Web Services