Release Notes for Grouper 2.3
Grouper v2.3.0 includes 25 fixes and improvements over v2.2.2. See the full list.
New Features in Grouper 2.3.0
Provisioning Service Provider Next Generation (PSPNG) implementation addresses performance problems and configuration complexity of the Provisioning Service Provider (PSP) | |
Grouper loader improvements: scheduling configuration stored in database facilitates high-availability changes to loader config files do not require restarting the loader, handling unresolvable subjects | |
| Web Service operations | New Web Service operations for attribute definitions, actions, and messaging |
| Grouper messaging | Grouper messaging system and WS with integration to the change log and ESB |
| UI screens | UI screens for attribute definitions and inherited privileges |
| Export to GSH | Export Grouper objects to a Grouper Shell (GSH) script |
| Folder privileges | Refactor folder privileges to be “admin” and “create” instead of “stem” and “create”. See the glossary for updated privileges and definitions. |
v2.3.0 patches
Grouper patching instructions (how to install patches with the Grouper installer)
note: if a java file is included in a patch, then all associated classfiles are there too
Patch | Description | Files affected |
|---|---|---|
GRP-1278: deadlock in grouper on upgrade (and maybe other times) | classes/edu/internet2/middleware/grouper/attr/assign/AttributeAssignBaseDelegate.java | |
classes/edu/internet2/middleware/grouper/app/gsh/obliterateStem.java | ||
GRP-1289: warning on new mysql driver and no ssl config in the connect url GRP-1290: built in stem for legacy attributes should use the root stem for built in objects | classes/edu/internet2/middleware/grouperClient/util/GcElUtilsSafe.java | |
classes/grouper.hibernate.base.properties | ||
GRP-1304: delete membership from UI has major performance problem | classes/edu/internet2/middleware/grouper/internal/dao/hib3/Hib3MembershipDAO.java | |
GRP-1308: grouper_ddl is slow due to selecting * from every table/view | classes/edu/internet2/middleware/grouper/ddl/GrouperDdlUtils.java | |
lib/log4j.jar | ||
lib/commons-lang.jar | ||
classes/edu/internet2/middleware/grouper/util/GrouperUtil.java | ||
GRP-1319: Use database metadata to see if a table or view exists | classes/edu/internet2/middleware/grouper/ddl/GrouperDdl.java classes/edu/internet2/middleware/grouper/ddl/GrouperDdlUtilsTest.java | |
GRP-1323: null pointer exception with inherited rule privilege and EL then clause | classes/edu/internet2/middleware/grouper/rules/RuleFinder.java | |
classes/edu/internet2/middleware/grouper/j2ee/ServletContextUtils.java | ||
GRP-1327: Handle case issue between subject source and loader source | classes/edu/internet2/middleware/grouper/app/loader/GrouperLoaderType.java | |
classes/edu/internet2/middleware/grouper/util/GrouperThreadLocalState.java | ||
classes/edu/internet2/middleware/grouper/audit/AuditTypeBuiltin.java | ||
classes/grouper.base.properties | ||
GRP-1303: null pointer exception in loader with default source id | classes/edu/internet2/middleware/grouper/app/loader/db/GrouperLoaderResultset.java | |
GRP-1343: Change log error when unassigning type and deleting type together | classes/edu/internet2/middleware/grouper/changeLog/ChangeLogTempToEntity.java | |
GRP-1352: GroupUniqueExtensionHook doesn't work when moving a group | classes/edu/internet2/middleware/grouper/hooks/examples/GroupUniqueExtensionHook.java | |
classes/edu/internet2/middleware/grouper/validator/DeleteStemValidator.java | ||
classes/edu/internet2/middleware/grouper/externalSubjects/ExternalSubjectSave.java | ||
GRP-1358: grouper config should use root stem for built in objects for all configs that need it | classes/grouper.base.properties | |
GRP-1365: loader threads can cause database pool timeout errors | classes/edu/internet2/middleware/grouper/app/loader/db/GrouperLoaderDb.java | |
classes/grouper.base.properties | ||
classes/edu/internet2/middleware/grouper/app/loader/GrouperLoader.java | ||
classes/edu/internet2/middleware/grouper/instrumentation/TierInstrumentationDaemon.java | ||
GRP-1381: Use member table subject identifier in loader jobs to improve performance | classes/edu/internet2/middleware/grouper/app/loader/LoaderMemberWrapper.java | |
classes/edu/internet2/middleware/grouper/Group.java | ||
classes/edu/internet2/middleware/grouper/changeLog/esb/consumer/EsbConsumer.java | ||
classes/grouper-loader.base.properties | ||
GRP-1365: loader threads can cause database pool timeout errors | classes/grouper.hibernate.base.properties | |
classes/edu/internet2/middleware/grouper/xml/export/XmlExportMembership.java | ||
GRP-1413: GroupSave with uuid and no name does not work for new uuids | classes/edu/internet2/middleware/grouper/attr/AttributeDefNameSave.java | |
GRP-1411: Generic error message when attempting to delete group that is part of a composite group | classes/edu/internet2/middleware/grouper/internal/dao/hib3/Hib3GroupDAO.java | |
GRP-1423: My memberships tab shows your memberships without taking into account security | classes/edu/internet2/middleware/grouper/privs/PrivilegeHelper.java | |
GRP-1417: migrate from grouper.ehcache.xml to hierarchical properties configuration | classes/ehcache.example.xml | |
classes/edu/internet2/middleware/grouper/util/GrouperUtil.java | ||
classes/edu/internet2/middleware/grouper/externalSubjects/ExternalSubjectSave.java | ||
classes/edu/internet2/middleware/grouper/attr/finder/AttributeDefNameFinder.java | ||
GRP-1439: remove records with a max number and loop so it doesnt fail | classes/edu/internet2/middleware/grouper/app/loader/GrouperLoaderType.java | |
GRP-1454: migrate from sources.xml to hierarchical properties configuration GRP-1452: regex replace in subject source can cause issues if subject id has dollar sign | lib/grouper/commons-digester.jar | |
GRP-1455 grouper versions do not match - GrouperVersion.class: 2.4.0 - greater than grouper.version.properties: 2.3.0 | classes/edu/internet2/middleware/grouper/misc/GrouperVersion.java | |
classes/edu/internet2/middleware/grouper/hibernate/ByHql.java | ||
GRP-1420: Member removal from group shows recent activity as GrouperSystem | classes/edu/internet2/middleware/grouper/hibernate/GrouperContext.java | |
GRP-1441: understanding diagnostic servlet output for loader jobs | classes/edu/internet2/middleware/grouper/j2ee/status/GrouperStatusServlet.java | |
classes/edu/internet2/middleware/grouper/app/loader/GrouperLoaderJob.java | ||
lib/grouper/commons-digester.jar | ||
GRP-1469: grouper check config is wrong for new digester jar | classes/edu/internet2/middleware/grouper/misc/GrouperCheckConfig.java | |
classes/edu/internet2/middleware/grouper/subj/SubjectHelper.java | ||
classes/edu/internet2/middleware/grouper/internal/dao/hib3/Hib3StemDAO.java | ||
classes/edu/internet2/middleware/grouper/app/loader/GrouperLoaderConfig.java | ||
classes/subject.base.properties | ||
classes/edu/internet2/middleware/grouper/attr/assign/AttributeAssign.java | ||
GRP-1445: Grouper loader real time updates - change log table for SQL jobs | classes/edu/internet2/middleware/grouper/internal/dao/hib3/Hib3DAO.java | |
classes/edu/internet2/middleware/grouper/app/loader/GrouperLoaderIncrementalJob.java | ||
GRP-1477: Reduce error logging when code fails in threads that would be retried out of threads | classes/edu/internet2/middleware/grouper/util/GrouperUtil.java | |
GRP-1286: Fix UI wording for v2.3.0 folder privilege changes | classes/grouperText/grouper.text.en.us.base.properties | |
GRP-1291: grouper ui text config should use superclass to implement local references to properties | classes/edu/internet2/middleware/grouperClient/config/GrouperUiTextConfig.java | |
classes/edu/internet2/middleware/grouper/ui/GrouperUiFilter.java | ||
classes/edu/internet2/middleware/grouper/grouperUi/serviceLogic/UiV2GroupImport.java | ||
GRP-1334: control links from new UI to admin and lite UIs via group | classes/edu/internet2/middleware/grouper/ui/GrouperUiFilter.java | |
GRP-1333: control which attributes are displayed for a subject and in which order | classes/grouper-ui.base.properties | |
GRP-1361: ui button to run loader job not showing up if "etc" is not stem for built in objects | classes/edu/internet2/middleware/grouper/grouperUi/beans/api/GuiGroup.java | |
GRP-1405: csrf error on grouper ui with no slash required token is missing from the request | classes/Owasp.CsrfGuard.overlay.properties | |
GRP-1411: Generic error message when attempting to delete group that is part of a composite group | classes/grouperText/grouper.text.en.us.base.properties | |
GRP-1423: My memberships tab shows your memberships without taking into account security | classes/edu/internet2/middleware/grouper/grouperUi/serviceLogic/UiV2MyGroups.java | |
classes/Owasp.CsrfGuard.overlay.properties | ||
classes/edu/internet2/middleware/grouper/ui/tags/GrouperComboboxTag2.java | ||
files/grouperExternal/public/assets/css/bootstrap.css | ||
GRP-1418: removeMembers redirect to main page for non-wheel user with Admin priv but not update | classes/edu/internet2/middleware/grouper/grouperUi/serviceLogic/UiV2Group.java | |
GRP-1463: what if you have a subject in your source identified by GrouperSystem | classes/edu/internet2/middleware/grouper/ui/GrouperUiFilter.java | |
classes/edu/internet2/middleware/grouper/grouperUi/beans/ui/ProvisioningContainer.java | ||
GRP-1317: attributeDefSave web service doesnt work for soap, GRP-1322: acknowledge message ws had an issue | classes/edu/internet2/middleware/grouper/ws/GrouperServiceLogic.java | |
GRP-1324: acknowledge message soap ws had an issue, GRP-1325: grouper WS send, receive, acknowledge message does not set Grouper headers | classes/edu/internet2/middleware/grouper/ws/coresoap/GrouperService.java | |
classes/edu/internet2/middleware/grouper/ws/GrouperServiceJ2ee.java | ||
GRP-1348: web services including hasMember doesnt not work when looking up group with alternate name | classes/edu/internet2/middleware/grouper/ws/coresoap/WsGroupLookup.java | |
classes/edu/internet2/middleware/grouper/ws/coresoap/WsExternalSubjectDeleteResults.java | ||
classes/edu/internet2/middleware/grouper/ws/coresoap/WsExternalSubjectToSave.java | ||
GRP-1447: Old version of mysql connector jar lurking in the V2.3.0 web service installation | lib/mysql-connector-java-5.1.5-bin.jar | |
GRP-1449: getGroups paging does not work if you dont pass in a stem | classes/edu/internet2/middleware/grouper/ws/GrouperServiceLogic.java | |
classes/edu/internet2/middleware/psp/grouper/PspChangeLogConsumer.java | ||
classes/edu/internet2/middleware/psp/grouper/PspChangeLogConsumer.java | ||
GRP-1318: PSPNG-Creation and Location of provision_to and do_not_provision_to | classes/edu/internet2/middleware/grouper/pspng/LdapAttributeProvisioner.java | |
classes/edu/internet2/middleware/grouper/pspng/Provisioner.java | ||
GRP-1392: Fixes slow group-selecting performance with large group registries | classes/edu/internet2/middleware/grouper/pspng/Provisioner.java | |
classes/edu/internet2/middleware/grouper/pspng/Provisioner.java | ||
GRP-1391 - Treats missing subjects in destination system as warnings instead of errors | classes/edu/internet2/middleware/grouper/pspng/LdapGroupProvisioner.java | |
classes/edu/internet2/middleware/grouper/pspng/Provisioner.java | ||
classes/edu/internet2/middleware/grouper/pspng/ProvisionerFactory.java | ||
GRP-1391: Failing on missing members (FullSync). Refactored the JexlExpression evaluation | classes/edu/internet2/middleware/grouper/pspng/Provisioner.java | |
GRP-1479 - Fix NullPointerExceptions, particularly when groups are deleted | classes/edu/internet2/middleware/grouper/pspng/LdapAttributeProvisioner.java | |
classes/edu/internet2/middleware/grouper/pspng/Provisioner.java |
For more information about upcoming plans, see the Grouper Product Roadmap .
Many other fixes and improvements were also made to all components of the Grouper Toolkit: Grouper API, Administrative & Lite UIs, Grouper Web Services, Grouper Client, Grouper Shell, Grouper Loader, PSP, and the Subject API.
Upgrading to Grouper 2.3 from Grouper 2.2
Using the Grouper Upgrader can simplify your upgrade process. Here is a movie demonstrating the Grouper upgrader. The upgrader can upgrade an installed env of the API, UI, WS, client, PSP, etc. If you dont have a build script to manage multiple envs, you might want to use the upgrader.
Important Changes in Grouper 2.3 that impact the upgrade
Inherited Privileges: The Grouper v2.3 UI has support for privilege inheritance. By default, if you are an admin on a folder, you can assign inherited privileges on it. Note that one potential side effect of this feature is that it allows end users to gain access to sub-folders and groups because they have admin access to a parent folder. In most cases, this is expected behavior because folders are typically delegated and managed hierarchically. However, if you do not allow parent folder admins to have access to all child objects, then you may want to disable this feature. You have the option to lock this feature down so only Grouper admins can use it or people in a certain group.
Other items before upgrading
- You may want to have your DBAs make sure you are not close to running out of tablespace. In general, it may be useful to have your DBAs available when you upgrade.
- If you have views that other systems use, you could replace them as tables before beginning.
- If you have other systems using Grouper, you could temporarily disable them.
Upgrade Steps
- You should get v2.3 versions of the Grouper API, Grouper UI, Grouper WS, Grouper Daemon, etc. from the Grouper Downloads page. You will need to merge configuration files and JARs.
Stop the Grouper Daemon. Once you prevent users from making updates to your Grouper instance, run the changeLogTempToChangeLog daemon to clear out the temp changelog using your existing v2.2 API. Here's an example using GSH.
gsh 0% loaderRunOneJob("CHANGE_LOG_changeLogTempToChangeLog")- Before performing any upgrade steps, export your Grouper registry. Options include performing a database backup (recommended) or using the XML Export utility in Grouper (not recommended since certain features may not get exported).
Using the 2.3 API, perform a registry check using GSH to create an SQL file that will contain the DDL to update your database. To do this, run: gsh -registry -check Note you may need to increase memory. For instance..
$ export MEM_MAX=2000m $ ./bin/gsh.sh -registry -check Using GROUPER_HOME: /opt/grouper Using GROUPER_CONF: /opt/grouper/conf Using JAVA: java using MEMORY: 64m-2000m Grouper starting up: version: 2.3.0, build date: 2016/04/20 16:15:04, env: <no label configured> grouper.properties read from: /opt/grouper/conf/grouper.properties Grouper current directory is: /opt/grouper log4j.properties read from: /opt/grouper/conf/log4j.properties Grouper is logging to file: /opt/grouper/logs/grouper_debug.log, /opt/grouper/logs/grouper_error.log, at min level INFO for package: edu.internet2.middleware.grouper, based on log4j.properties grouper.hibernate.properties: /opt/grouper/conf/grouper.hibernate.properties grouper.hibernate.properties: sa@jdbc:hsqldb:hsql://localhost:9001/grouper sources.xml read from: /opt/grouper/conf/sources.xml sources.xml groupersource id: g:gsa sources.xml groupersource id: grouperEntities sources.xml jdbc source id: jdbc: GrouperJdbcConnectionProvider This db user 'sa' and url 'jdbc:hsqldb:hsql://localhost:9001/grouper' are allowed to be changed in the grouper.properties Continuing... Grouper ddl object type 'Grouper' has dbVersion: 29 and java version: 30 Grouper database schema DDL requires updates (should run script manually and carefully, in sections, verify data before drop statements, backup/export important data before starting, follow change log on confluence, dont run exact same script in multiple envs - generate a new one for each env), script file is: /opt/grouper/ddlScripts/grouperDdl_20160420_16_15_53_708.sql Note: this script was not executed due to option passed in To run script via gsh, carefully review it, then run this: gsh -registry -runsqlfile /opt/grouper/ddlScripts/grouperDdl_20160420_16_15_53_708.sql
- In this example above, an SQL script called /opt/grouper/ddlScripts/grouperDdl_20160420_16_15_53_708.sql was created.
- Postgres only - If using postgres, you should see foreign keys being dropped at the top of the script. If not, try setting the ddlutils.schema grouper.properties setting and run again. If you still don't see foreign keys being dropped at the top of the script, manually drop all foreign keys before running the script.
- Postgres and hsql only - You should backup any non grouper views that depend on Grouper views, run the grouper script (which deletes those views due to drop view cascade), and then you should recreate those non grouper views.
Run the SQL script.
If you are running via GSH, make sure this is in log4j.properties so that you know which line of the script is currently executing to see progress and troubleshootlog4j.logger.org.apache.tools.ant = WARN
To do this, run: gsh -registry -runsqlfile /path/to/sql/file.sql For instance..$ ./bin/gsh.sh -registry -runsqlfile /opt/grouper/ddlScripts/grouperDdl_20160420_16_15_53_708.sql Using GROUPER_HOME: /opt/grouper Using GROUPER_CONF: /opt/grouper/conf Using JAVA: java using MEMORY: 64m-2000m This db user 'sa' and url 'jdbc:hsqldb:hsql://localhost:9001/grouper' are allowed to be changed in the grouper.properties Continuing... Script was executed successfully Grouper starting up: version: 2.3.0, build date: 2016/04/20 16:15:04, env: <no label configured> grouper.properties read from: /opt/grouper/conf/grouper.properties Grouper current directory is: /opt/grouper log4j.properties read from: /opt/grouper/conf/log4j.properties Grouper is logging to file: /opt/grouper/logs/grouper_debug.log, /opt/grouper/logs/grouper_error.log, at min level INFO for package: edu.internet2.middleware.grouper, based on log4j.properties grouper.hibernate.properties: /opt/grouper/conf/grouper.hibernate.properties grouper.hibernate.properties: sa@jdbc:hsqldb:hsql://localhost:9001/grouper sources.xml read from: /opt/grouper/conf/sources.xml sources.xml groupersource id: g:gsa sources.xml groupersource id: grouperEntities sources.xml jdbc source id: jdbc: GrouperJdbcConnectionProvider Grouper note: auto-created stem: etc:attribute:messages Grouper note: auto-created role: etc:attribute:messages:grouperMessageRole Grouper note: auto-created attributeDef: etc:attribute:messages:grouperMessageTopicDef Grouper note: auto-created attributeDef: etc:attribute:messages:grouperMessageQueueDef Grouper note: auto-created stem: etc:attribute:messages:grouperMessageTopics Grouper note: auto-created stem: etc:attribute:messages:grouperMessageQueues
Note that if one of the SQL statements in the script fails, the process will abort leaving the rest of the SQL statements from executing. If this happens, in most cases, you can't just re-run the full script since re-executing some of the DDL changes that previously succeeded would fail now (e.g. dropping a view or constraint that was previously dropped successfully.) You could edit the script to remove the statements that previously succeeded in order to re-execute the statement that failed and the ones after it. Or you can run the previous step again to generate a new SQL script.
Now that the DDL updates have been made, there is an additional GSH command that needs to be run. To do this, run: gsh ../misc/postGrouper2_3_0Upgrade.gsh (The gsh script is in the "misc" directory.) Note you should check the output to make sure no errors are thrown. If you see an error, it is safe to re-run. For instance..
$ ./bin/gsh.sh misc/postGrouper2_3_0Upgrade.gsh Using GROUPER_HOME: /opt/grouper Using GROUPER_CONF: /opt/grouper/conf Using JAVA: java using MEMORY: 64m-2000m Grouper starting up: version: 2.3.0, build date: 2016/04/20 16:15:04, env: <no label configured> grouper.properties read from: /opt/grouper/conf/grouper.properties Grouper current directory is: /opt/grouper log4j.properties read from: /opt/grouper/conf/log4j.properties Grouper is logging to file: /opt/grouper/logs/grouper_debug.log, /opt/grouper/logs/grouper_error.log, at min level INFO for package: edu.internet2.middleware.grouper, based on log4j.properties grouper.hibernate.properties: /opt/grouper/conf/grouper.hibernate.properties grouper.hibernate.properties: sa@jdbc:hsqldb:hsql://localhost:9001/grouper sources.xml read from: /opt/grouper/conf/sources.xml sources.xml groupersource id: g:gsa sources.xml groupersource id: grouperEntities sources.xml jdbc source id: jdbc: GrouperJdbcConnectionProvider Type help() for instructions Error: Cannot properly read UTF string from resource: grouperUtf8.txt: 'ٹٺٻټكلل' ########################################## # Grouper 2.3.0 Upgrade Step 1/1: Remove grouperLoaderLdapErrorUnresolvable attribute ########################################## edu.internet2.middleware.grouper.GrouperSession: 5868a5370afd4941bf3f340bf632546f,'GrouperSystem','application' edu.internet2.middleware.grouper.attr.AttributeDefName: AttributeDefName[name=etc:attribute:loaderLdap:grouperLoaderLdapErrorUnresolvable,uuid=799596896dd0426fb4c4e8edf9bd8a98] Successfully removed attribute.
- Analyze your tables. (To avoid any performance issues later.)
- Start the Grouper Daemon and all other Grouper components (UI/WS).
- The Grouper member table (grouper_members) now has a new column to store subject identifiers. Post 2.3.0, this will be used to help improve Grouper's performance in various aspects. You will need to configure your sources.xml file and sync the new column. For details: Subject Identifier column in member table
- By default any folder owner can assign inherited privileges, which means they can get control of any descendant object. If you do not want this you can lock down control
See Also
See Also
regex replace in subject source can cause issues if subject id has dollar sign