Attending
- Chris Hyzer, Penn, Chair
- Shilen Patel, Duke
- Chad Redman, University of North Carolina Chapel Hill
- Vivek Sachdiva, independent
- Jeff Williams UNCG
- Carey Black, the Ohio State University
- Emily Eisbruch, Internet2
Administrivia
- https://internet2.edu/community/about-us/policies/internet2-intellectual-property-policy/
- Approve minutes
- Review AIs Grouper Project Action Items (Google Doc)
- Agenda bash
New Action Item from this call
- AI Shilen - Create a JIRA and assign to Chris Hubing for adding metadata for registration to Grouper Demo Site Access
- AI Chris - look at how to use the UI text for localized error messages
Discussion
Roadmap
- Grouper roadmap engagement with the community is being worked on
- SteveZ will provide best methods for community outreach around Grouper 3.0
- Ideas:
Next 6 months, finish Grouper 2.5, including - Provisioners
- Improved performance
- Next generation of subject sources
- Migration of subject sources
- Next version will be Grouper 3.0
- Ask community for medium sized tasks needed in Grouper
- Wait for Grouper 3.0 to add new features
- Example: secure file for report
- Membership not hitting failsafe limits
- Example: secure file for report
- One month fixing JIRAs
- Message will be:
- Grouper 2.5 will be long term support
- Grouper 2.5 is the supported version
- Monthly releases of 2.5 for bug fixes, low risk upgrade
- Please upgrade to it
- Migrate to new subject sources and new provisioning
- Grouper 2.5 will be long term support
- Decision around migrating to postgres
- Better performance
- Document how to run postgres for those who dont run it
- Performance benchmarks
- Better performance
- Database is using too much resource
- We evolved the Grouper database with many layers
- Use less memory in the tomcats
- Change existing tables or make shadow tables
- Same challenge with indexes, if we start over with structure it can become more efficient
- Multiplicity of queries due to need to run tests
- We can become more effective
- Cache things and clear out cache for testing
- Make production more efficient
- Are we supporting batching efficiently?
- Make sure things are batchable
- Memberships and attribute assignments are most important
- Like Shilen improved the changelog, using batching
- Do we want to keep hibernate?
- Adds overhead
- Does Clobs and blobs seamlessly
- Does paging well
- That may not be an issue if we go to Postres
- May not want to move away from Hibernate
- Move away from ehcache?
- Be more mindful of when we are using ehcache?
- Privileges , using a lot of chaining code
- Web Service w JSON
- Soap XML can be off by default
- Grouper Client is using XML , can switch to JSON
- Get rid of old libraries with issues
- Consider items that would make a big improvement
- question: what would replace hibernate with?
- Look for something lighter weight
- Need to research this
- Nested groups
- Query planner issues
- Grouper fields table is because we used to have custom lists
- Memberships table may be used for too many items
- Split things out into single purpose tables may help with performance
- UI issues, so many pixels wide
- Challenging for visualization
- Upgrade all libraries… should be part of build/release cycle, should be automated
- Shilen: Some UI improvements may be possible prior to Grouper 3.0
- Table redesign, the views would still work as is?
- Yes , or perhaps have some Grouper v3 views that are more efficient
- Handling of UUID
- Keep legacy working and also evolve
- Provide a way to migrate over time
- One database only could be an issue for some sites
- Backups with postgres require some support
- Matt: Folder privacy is important
- Suggestion to have folder privilege
- Set of hidden groups where each group maps to a folder
- If you add a folder privilege, leave opportunity to use loaders
- We will need to delve into the best approach
- If you add a folder privilege, leave opportunity to use loaders
- JasperReports has stand alone web server now
- Present an outward facing view with priv based logic?
- Add a reporting engine bolt on
- Making all existing audit report things work
- Documentation improvements
- Matt: GITLAB, moving away from wiki
- Helps with version control
- Jeffrey: migration from Oracle to Postgress could be an issue for some
- Cosmetic issues with the Grouper UI
Current Work
Vivek
- Worked on object types
- Attributes on stem, more consistent model
- Deprovisioning and attribute propagation will happen in background
- Shilen worked on minimizing number of queries
- Improving efficiency
- Changelog consumer controls the process
- Provisioning is special case since there are sync tables
- Request for Chris to review Vivek’s work
-
- Now working on Custom UI
- The community has not taken advantage of Custom UI yet
- JSON and attributes on groups is a speed bump
- Need to migrate how Custom UI works
- Use Grouper Config, like GSH template works,
- Model config of custom UI and have a wizard
- Meta Config into properties file
- Migration utility to take JSON and migrate to properties
Chris
- Did a build, Maven worked
- Hope to get build out today
- 70 Jiras
- GSH template fixes
- Some reporting fixes
- Manage interface for VPNs
- Adding more validation to provisioning
- First level of validation is stuff in JSON and config file
Shilen
- Made performance adjustments
- LDAP DAO was getting repeated queries, resolved that
- Code as is now, when you go into UI and set something as provisionable, there is direct assignment, gets propagated, changelog runs as part of provisioner, replicates to group sync table
- UI will still allow provisionable, but skip setting in attribute framework,
- Gets set in sync tables
- Changed to return what propagation should be
- Makes LDAP full sync test pass
- Will work on metadata issues and incremental
- Need to add a column in Grouper sync table for groups for metadata
- How does that fit into group propagation?
- Chris: If we keep the member and membership metadata in attributes, since no indirect assignments, then we are OK
- Shilen will focus on groups, not on membership
- To add column in Grouper sync group table,
- Chris has a wiki on DDL changes that will help
- DDL and Grouper v2.5
- Shilen: Work again on DN overrides
Chad
- Doing Grouper projects for UNC
- GSH Templates on mock database
- Used InteliJ so aware of Grouper libraries
- Worked well, has not gone out to actual environments yet
- Provisioning work: less straightforward
- Lining up attributes was an issue
- Issue on provisioning: multiple types you want to provision, person type or application types.
- About 50K subjects
- Design issue: if you have multiple subject types to provision, all LDAP, you can have person types UID= or application types, CN=, with different subject sources and different checkboxes, how would you look them up, you just have single field,
- What Logic to use for this
- Need to think of logic, perhaps related to subject source
- Documentation is conceptual, it not explicitly telling you what “you need this in this field”
- Name field is DN… must calculate it
- Hard to know what’s a bug, issue of being an early adopter
- Issue: have read-only version of LDAP for subject queries
- Not accessible by the UI
- Hard to test, see GRP 3402
- Test config issue
- Fix is: hard code error messages?
- For localized error messages, don’t tie them to the UI
- Should GSH have access to the Jar?
- Must create fake HTTP Server request
- AI Chris will look at how to use the UI text for localized error messages
- Chad’s Moonshot items for Grouper 3.0 and beyond
- Web services should be beefed up, quirky way of doing REST
- Consider swagger
- TOP ISSUES FOR CHAD
- Different subject sources and different look ups for entities
- GRP-3402
LdapGrouperExternalSystem remove dependence on UI - Want to be able to look up subjects individually as opposed to all of them
- Blank as a member
Issue Roundup
Jiras in past two weeks
- GRP-3434
override dn of group in ldap provisioning
GRP-3433
attributeDefName extensions should not contain special characters by defaults
GRP-3432
attributeDef extensions should not contain special characters by defaults
GRP-3431
stem extensions should not contain special characters by defaults
GRP-3430
group extensions should not contain special characters by defaults
GRP-3429
pspng should add insert count to hib3loaderlog and not replace
GRP-3428
pspng gets stuck on deletes or cases where the group is null
GRP-3427
htmlescape recent activity for configuration changes
GRP-3426
do not allow attributeDefNames with same name and different case
GRP-3425
do not allow attributeDefs with same name and different case
GRP-3424
do not allow groups with same name and different case
GRP-3423
do not allow stems with same name (case insensitive) by default
GRP-3422
validate provisioning entityAttributes has a membership attribute
GRP-3421
validate provisioning groupMemberships has a membership attribute
GRP-3420
validate that entityAttributes provisioning has a matching entity attribute
GRP-3419
validate that groupMemberships provisioning has a matching group attribute
GRP-3418
if you are inserting groups/entities, then dn should insert
GRP-3417
if selecting groups or entities then you must select the dn
GRP-3416
if operating on groups or entities, then dn is required
GRP-3415
if operating on groups/entities/memberships, must either select or insert
GRP-3414
validate that provisioner is doing something with groups, entities, or memberships
GRP-3413
validate provisioner specific validation on provisioner save
GRP-3412
recentMembership jobs should not be subject to FailSafe logic
GRP-3411
GSH templates ( UI ) should support an input type of "Subject picker"
GRP-3410
other input’s available for replacement in the SQL string
GRP-3409
when adding a group (e.g. with app template) do we need two audits, one for add one for edit?)
GRP-3408
update rabbitmq tls version
GRP-3407
if the ldap provisioning group name in groupAttributes is not translated, but has a group link, it should copy from the sync table
GRP-3406
clear out error codes in sync provisioning objects before printing in logs or diagnostics
GRP-3405
provisioning only validate fields for update during update, insert during insert
GRP-3403
keep a bad subject log
GRP-3402
LdapGrouperExternalSystem remove dependence on UI
GRP-3401
grouper should not allow same object type with same case insensitive name by default
GRP-3400
add dropdown for ldap provisioning group rdn
GRP-3399
auto-configure an ldap external system test by finding the username (object scope)
GRP-3398
add ability to export non-base config from ui for a certain config file
GRP-3397
non folder admins cannot retrieve folder reports
GRP-3396
reports on groups cannot be retrieved from email by non admins
GRP-3395
gsh templates can require input type, should default to string in all cases
GRP-3394
people who can run templates (but cant create in folder) should see template menu items and run template
GRP-3393
attestation stem save should be able to update attestation date for reports
GRP-3392
show pre-template errors on screen in gsh template if configured to show
GRP-3391
Grouper Provisioning - performance issue when checking value types for memberships
GRP-3390
Grouper Provisioning target dao retrieve entities duplicates
GRP-3389
save group (and stem?) should set parent display extensions if creating them
GRP-3388
add ability to have dynamic values for gsh template inputs
GRP-3387
create a function with url to compare two groups
GRP-3386
add exclusive or composite type
GRP-3385
object type propagation rewrite
GRP-3384
stem attestation screen form has non bold label
GRP-3383 gsh template error message is misleading (about membership import)
GRP-3382 custom ui gsh exec should throw exception from gsh
GRP-3381 typo in AttributeAssignToAssignmentSave checking type of attribute (multi assign or not)
GRP-3380 improve group/stem report output
GRP-3379 improve GSH template redirect after run
GRP-3378 showEl on gsh templates has issues on hidden field
GRP-3377 group/stem reports on daemon screen show innocuous error
GRP-3376 improve AttestationStemSave
GRP-3375
add manual and intermediate types to visualization legend
GRP-3374
add sql dropdowns to gsh templates based on user
GRP-3373
add useful imports to gsh groovy profile (since compiles are cached) and make them java friendly
GRP-3372
add ability to call a gsh template from another template and consolidate output
Grouper Emails in past two weeks
- Re: [grouper-users] Grouper 2.4 upgrade deleted some groups from Active Directory, Hyzer, Chris, 04/14/2021
- [grouper-users] Slack channel?, Maiko Lehman, 04/14/2021
- [grouper-users] LDAP Grouper Loader paging issue, T-Heetderks, 04/23/2021
- Re: [grouper-users] LDAP Grouper Loader paging issue, Shilen Patel, 04/23/2021
Grouper wiki updates in past two weeks
- Note from Chris Hubing re RabbitMQ Messaging wiki page: https://spaces.at.internet2.edu/display/Grouper/Grouper+Messaging+with+RabbitMQ?focusedCommentId=190351093#comment-190351093
- v2.5 Upgrade Instructions from v2.5
- Grouper Custom UI
- Visualization UI
- v2.5 Release Notes
- GrouperShell (gsh)
- Get Groups
- Grouper UI templates
- Grouper custom template via GSH invoked by daemon - load group attributes
- Grouper daemon "other job" to run a script
- Grouper custom template via GSH invoked by daemon - load group attributes
- Grouper custom template via GSH
- Grouper GSH template security
- How to Setup a lite Grouper Development Environment for Grouper v2.5