has been upgraded to Confluence 6.6.0. If you have any questions and/or concerns, please contact us at
Child pages
  • Grouper Provisioning: PSPNG
Skip to end of metadata
Go to start of metadata

Provisioning's job is to reflect Groups and their Memberships in other systems. This is handled in Grouper via the PSPNG component, which is mostly a client of the Grouper changelog and is designed as a ChangeLog Consumer (CLC). Detected changes in Grouper via the changelog are picked by PSPNG and evaluated for provisioning operations selectively.


Over the years, dozens of provisioners have been created -- some focused on a single destination type and others with some generic functionality combined with a very wide variety of options and capabilities.

In recent years, the Grouper Team and Users have concluded that the provisioning priorities should change: less flexibility and increased simplicity and performance. The Next Generation of the Grouper Provisioning Service Provider (aka PSPNG) was defined in Post PSP Provisioning and is expected to replace the older PSP components over time.

Development Status and Features

PSPNG's general structure should be ready to provision various targets, but its current implementation is limited to provisioning LDAP targets like:

  • -(Unix) LDAP Groups: (GroupOfUniqueNames, GroupOfNames, PosixGroup)
  • -Active Directory Groups
  • -LDAP Attributes (like eduPersonEntitlement)


PSPNG's Configuration is done via the conf/ file, in the grouper API Binary, with a paragraph for each provisioning destination, as well as an additional paragraph that enables and configures FullSync operation. There are several configuration options documented in the following spreadsheet:

Provisioner TypeParameterDefaultDescriptionDefault Behavior
All Provisioners

Jexl expression that refers to stem_attributes, group_attributes, or groupProvision groups if <provisionerName> is in a group or stem provision_to attribute AND NOT in a do_not_provision_to attribute

600 (seconds)
How long should Grouper (Group, Stem, Subject) data be cached by the provisioners?Grouper data will be cached for 10 minutes, though it is flushed when groups change.

10000 (Groups)
How many Grouper Groups should be kept in memory at a time?

10000 (Subjects)
How many Grouper Subjects should be kept in memory at a time?

Does provisioniner need User/Subject information from the Target System? For example, do any JEXL expressions need information that is not available in Grouper Subjects?All provisioning will be done based (only) on Grouper-Subject information.

Does provisioner need group information from the Target System? For example, this information could be used in various JEXL expressions.All provisioning will need to be done based on Grouper-Group information

Only used when needsTargetSystemUsers=true: Should users be created when they cannot be found?Users will not be created by Grouper Provisioning. Provisioning actions that require the users will fail.

Only used when needsTargetSystemUsers=true: How many users can be sought in a single Fetch?Fetches will seek information for up to 50 users at a single time.

Only used when needsTargetSystemGroups=true: How many groups can be sought in a single Fetch?Fetches will seek information for up to 50 groups at a time.

Can groups be created without any members? If so, it is easier to create them separately from membership changes.Yes, create groups as soon as possible.

FullSync: Wait a bit before retrying a group that has failed. This prevents aggressive infinite loops.1 second pause before retrying a failed group.
LdapProvisioner (Abstract)
What ldap pool should be used by this provisioner

Where to find users?Required if provisioner needsTargetSystemUsers=true

Jexl expression that refers to stem_attributes, group_attributes, or groupHow to find users in the Target System?

Comma-separated list of attributes that are useful for logging and that are needed by userSearchFilter or by a subclass's ValueFormatsReads common attributes from either Unix or ActiveDirectory LDAP servers

Whether paging should be enabled in LDAP search requests; This typically requires the paging extension to be enabled and configured in LDAP to avoid LDAP Error Code 12.TRUE

How many result objects can be pulled by a single request. This is small to avoid problems by default.Break the results of a large query into fairly tiny chunks.

How long to keep User information in memory?Keep User information in memory for 10 minutes, though user-information is flushed when users are changed by a provisioner

How many LDAP accounts can be kept in memory at a time, indexed by the Subject mapped to them?Keep the last 10000 users found by searching with Subject information

Is this an active-directory server? If so, then AD-specific attribute-value-paging is enabled. Also, member (reverse user-to-group virtual attribute) is enabled.LDAP server is treated like a non-active-directory server. Problems will occur with full-sync of large groups.

How many values can be added/removed from an attribute in a single ldap operationBreaks large list of values that need to be added/removed from an attribute into chunks that this size. For example, 5000 values that need to be added would be added in 50 chunks of 100 values each.
LdapGroupProvisioner (Also Provisioner and LDAPProvisioner)
'member' for AD
<required> otherwise
What attribute represents a group's members in the Target System?Active Directory should just work. Otherwise, this is required.

What value (typically based on Subject or TargetSystemUser information) is written into the memberAttributeName attribute of groups?

Active Directory and GroupOfUniqueNames will typically work. This is a JEXL expression and is parsed at runtime. You may choose to script your way into this, by perhaps choosing a specific attribute:


memberof for AD
null otherwise
Virtual attribute of accounts that lists their groups

What LDIF should be written to the directory to add a group. Multiple lines need to be separated by || (double-pipes). The DN of the LDIF will be combined with groupCreationBaseDn>

Where should groups be created? At group-creation time, this is appended to the DN that results from the groupCreationLdifTemplate.Groups are created starting at the top of the search BaseDn.

Where are groups found?

Should groups in the groupSearchBaseDn/allGroupSearchFilter be removed if they no longer exist in Grouper?

FUTURE: How to find all the groups that grouper-provisioning maintains. If <grouperIsAuthoritative>, then groups found via this filter will be removed during a full sync.Groups are not removed when they are removed from Grouper nor when they no longer match the groupSelectionExpression.

How to find a group, based on Grouper Group (or stem) information

Attributes that should be read from groups when searching for them. This needs to include all the attributes used in singleGroupSearchFilter. This should not include the attribute which holds the group's members.Support common, basic singleGroupSearchFilters.

How long should LDAP-Group information be cached in memory?Keep LDAP Group information in memory for 10 minutes, though it is flushed when users are changed by a provisioner.

How many LDAP groups to keep in memory, indexed by Grouper Group.

See above (JEXL expressions use User and Group information from the Target System)

See above (JEXL expressions use User and Group information from the Target System)
(Also Provisioner and LDAPProvisioner)
What attribute is changed in User LDAP objects to represent group membership?

What value (typically based on Subject or TargetSystemUser information) is written into the provisionedAttributeName users?The stem:Group name is written to the attribute specified in <provisionedAttributeName>

See above (JEXL expressions only use User information from the Target System)

See above (JEXL expressions only use User information from the Target System)

What values of the attribute is grouper authoritative for during a full sync? null (default) or empty means that pspng will only process removals as memberships change, and won't clean up unknown attribute values.Warning: Grouper should have full control over the target attribute to avoid complications that come from sharing attributes with multiple provisioning tools.

LDAP Properties

LDAP configuration is done based on the ldaptive library's property configuration. A paragraph of ldap configuration is created in for each LDAP endpoint, and that paragraph is referenced by the appropriate provisioners.  Comments in refers to these "LDAP pools" as "LDAP connections".

At this time, the LDAP bindCredential cannot be encrypted via the Grouper morphstring.

ldap.groupOfNames.ldapUrl = ldaps: //hostname
ldap.groupOfNames.bindDn = cn=xxxxxx,ou=xxxxx
ldap.groupOfNames.bindCredential = xxxxx
ldap.groupOfNames.someOtherLdapProperty = value

# in another paragraph a provisioner is associated with the LDAP pool name
changeLog.consumer.pspng_groupOfUniqueNames.ldapPoolName = groupOfNames

Advanced Ldaptive Properties

PSPNG relies on the Ldaptive library for all LDAP-related operations. To learn more about what other LDAP properties are available,  one simple example can be found here. Moving into more realistic examples will probably be helped by looking at the ldaptive configuration classes and the setters available within them: connectionspooling, binding (sasl, gssapix509, jks, etc).

In case it is helpful, this is currently implemented here. You may also wish to take a look at GRP-1306 to learn more about the differences between vtdap (the previously used LDAP library) vs ldaptive.


PSPNG runs as part of the grouper loader. So simply run the loader with:

cd <grouper-api-binary-folder>/bin
./gsh -loader

Group/Folder Selection

Groups can be enabled and disabled based on Group or Folder/Stem information and attributes. By default, two multivalued attributes are used:

Attribute DefinitionAttribute NameDescription


etc:pspng:provision_toEnable downstream provisioning


etc:pspng:do_not_provision_toDisable downstream provisioning

How do I create PSPNG attribute definitions?

These attribute definitions are auto-created by Grouper the very first time PSPNG runs. If you have configured PSPNG to run on startup, then the attributes will be created then or else you may need to wait for the scheduled PSPNG job to kick in.

If you need to, you can always create these via the Grouper Shell. 

How do I assign PSPNG attribute definitions?

These attributes need to be assigned to Groups or Folders via the Lite UI. 

After the attribute is assigned to a folder or group, you need to assign it a value. The value of the attribute  MUST  match the provisioner name that is defined in the configuration (i.e. pspng_groupOfUniqueNames).

PSPNG will evaluate whether a group or stem/folder qualifies for provisioning by running the group-selecting filter as a JEXL expression. The expression is able to process and evaluate attribute definitions on a given group/folder and returns true/false to indicate whether PSPNG should continue with downstream provisioning.

Logging & Troubleshooting

To troubleshoot PSPNG configuration, you may need to add the following logging configuration to the file:

Account creation

To facilitate (load) testing, Grouper PSPNG has limited ability to create accounts in a target system. This ability is very rudimentary and should not be used to maintain accounts in production; such maintenance should be done by an IdM product or some other ETL mechanism. If you choose to ignore this best practice, you'll probably run into two main problems:

  1. Grouper/PSPNG has no ability to keep accounts up to date; it only creates them if they're missing, and
  2. Your Grouper-subject mappings probably don't have some of the information needed for account provisioning. And, even if all you need is name, username, and email address (which are probably in your subject mappings), you'll still run into problem (1) where Grouper offers no mechanism to update name and email address when they change in your subject source.

Full SYNC Provisioning

Upgrade warning (PSPNG Patch #11, June 8, 2017)

The Full-Sync-provisioning improvements in Grouper 2.3/PSPNG Patch #11 are configured with new, better properties. (You should no longer use the properties that start with changeLog.psp.fullSync.) 

In the background of PSPNG, there is always a full-sync-provisioning engine running which is automatically used when incremental provisioning finds conflicting changes or otherwise is unable to handle the changelog events. The full-sync items in do not alter/configure the background engine but instead define quartz jobs that send all the groups marked for provisioning into the queues that drive the full-sync engine. Additionally, if a provisioner is "authoritative" (changeLog.consumer.<provisioner>.grouperIsAuthoritative=true), then a cleanup task is included in the scheduled job which will delete extra groups or attributes that no longer exist in the Group Registry. (Note: Data in a provisioning target will always get deleted by incremental provisioning when groups are deleted, regardless whether a provisioner is authoritative. Full-sync cleanup is just a safety net in case incremental provisioning misses something, or if the data was written into the target system outside of PSPNG.)

Each provisioner that wants to schedule periodic full-syncs will need the following lines included in

Full Sync Provisioner
otherJob.<provisioner-name>_full.class = edu.internet2.middleware.grouper.pspng.FullSyncStarter
otherJob.<provisioner-name>_full.quartzCron = 0 0 0 * * ?   #Every midnight

# Note, there is presently no "runAtStartup" option as there was before PSPNG Patch #11. Please see GRP-1563.

<provisioner-name> is the 3rd field of the properties used to configure the rest of the provisioner. For instance, <provisioner-name> is xyz for the following config snippet: = edu.internet2.middleware.grouper.pspng.PspChangelogConsumerShim = edu.internet2.middleware.grouper.pspng.LdapGroupProvisioner

Provisioner Templates

The syntax for each line is:

changelog.consumer.<provisioner-name>.<propertyName> = value

Bushy DNs

In the provisioner snippets below, the groups will be created hierarchically.  To get flatter group names, you can use the utils.bushyDn function.  For example:

changeLog.consumer.<provisioner-name>.groupCreationLdifTemplate = dn: cn=${},${utils.bushyDn(, "cn", "ou")},dc=example,dc=edu


This is most likely the provisioner needed for most LDAP servers, such as OpenLDAP, OpenDJ, etc.
changeLog.consumer.pspng_groupOfUniqueNames.class = edu.internet2.middleware.grouper.pspng.PspChangelogConsumerShim
changeLog.consumer.pspng_groupOfUniqueNames.type = edu.internet2.middleware.grouper.pspng.LdapGroupProvisioner
changeLog.consumer.pspng_groupOfUniqueNames.quartzCron = 0 * * * * ?
changeLog.consumer.pspng_groupOfUniqueNames.ldapPoolName = opendj
changeLog.consumer.pspng_groupOfUniqueNames.memberAttributeName = uniqueMember
# changeLog.consumer.pspng_posixGroup.memberAttributeValueFormat = ${ldapUser.getStringValue("uid")}
changeLog.consumer.pspng_groupOfUniqueNames.memberAttributeValueFormat = ${ldapUser.getDn()}
changeLog.consumer.pspng_groupOfUniqueNames.groupSearchBaseDn = ou=grouper,ou=groups,dc=example,dc=edu
changeLog.consumer.pspng_groupOfUniqueNames.allGroupsSearchFilter = objectclass=groupOfUniqueNames
changeLog.consumer.pspng_groupOfUniqueNames.singleGroupSearchFilter = (&(objectclass=groupOfUniqueNames)(cn=${}))
changeLog.consumer.pspng_groupOfUniqueNames.groupCreationLdifTemplate = dn: cn=${}||cn: ${}||objectclass: groupOfUniqueNames
changeLog.consumer.pspng_groupOfUniqueNames.userSearchBaseDn = cn=users,dc=example,dc=edu
changeLog.consumer.pspng_groupOfUniqueNames.userSearchFilter = uid=${}


changeLog.consumer.pspng_posixGroup.class = edu.internet2.middleware.grouper.pspng.PspChangelogConsumerShim
changeLog.consumer.pspng_posixGroup.type = edu.internet2.middleware.grouper.pspng.LdapGroupProvisioner
changeLog.consumer.pspng_posixGroup.quartzCron = 0 * * * * ?
changeLog.consumer.pspng_posixGroup.ldapPoolName = opendj
changeLog.consumer.pspng_posixGroup.memberAttributeName = memberUid
changeLog.consumer.pspng_posixGroup.memberAttributeValueFormat = ${ldapUser.getStringValue("uid")}
changeLog.consumer.pspng_posixGroup.groupSearchBaseDn = ou=grouper-posix,ou=groups,dc=example,dc=edu
changeLog.consumer.pspng_posixGroup.allGroupsSearchFilter = objectclass=posixGroup
changeLog.consumer.pspng_posixGroup.singleGroupSearchFilter = (&(objectclass=posixGrouper)(cn=${}))
# Obviously, gidNumber should be based on a grouper-group attribute
changeLog.consumer.psong_posixGroup.groupCreationLdifTemplate = dn: cn=posix-${}||cn: posix-${}||objectclass: posixGroup||objectclass: groupOfNames||gidNumber: ${group.idIndex}
changeLog.consumer.pspng_posixGroup.userSearchBaseDn = cn=users,dc=example,dc=edu
changeLog.consumer.pspng_posixGroup.userSearchFilter = uid=${}


changeLog.consumer.pspng_activedirectory.class = edu.internet2.middleware.grouper.pspng.PspChangelogConsumerShim
changeLog.consumer.pspng_activedirectory.type = edu.internet2.middleware.grouper.pspng.LdapGroupProvisioner
changeLog.consumer.pspng_activedirectory.quartzCron = 0 * * * * ?
changeLog.consumer.pspng_activedirectory.ldapPoolName = active_directory
changeLog.consumer.pspng_activedirectory.isActiveDirectory = true
changeLog.consumer.pspng_activedirectory.memberAttributeName = member
changeLog.consumer.pspng_activedirectory.memberAttributeValueFormat = ${ldapUser.getDn()}
changeLog.consumer.pspng_activedirectory.groupSearchBaseDn = ou=grouper,ou=groups,dc=example,dc=edu
changeLog.consumer.pspng_activedirectory.allGroupsSearchFilter = objectclass=group
changeLog.consumer.pspng_activedirectory.singleGroupSearchFilter = (&(objectclass=group)(cn=${}))
changeLog.consumer.pspng_activedirectory.groupCreationLdifTemplate = dn: cn=${}||cn: ${}||objectclass: group
changeLog.consumer.pspng_activedirectory.userSearchBaseDn = cn=users,dc=example,dc=edu
changeLog.consumer.pspng_activedirectory.userSearchFilter = samAccountName=${}


changeLog.consumer.pspng_attributes.class = edu.internet2.middleware.grouper.pspng.PspChangelogConsumerShim
changeLog.consumer.pspng_attributes.type = edu.internet2.middleware.grouper.pspng.LdapAttributeProvisioner
changeLog.consumer.pspng_attributes.quartzCron = 0 * * * * ?
changeLog.consumer.pspng_attributes.retryOnError = true
changeLog.consumer.pspng_attributes.ldapPoolName = opendj
changeLog.consumer.pspng_attributes.provisionedAttributeName = eduPersonEntitlement
changeLog.consumer.pspng_attributes.provisionedAttributeValueFormat = g:${}
changeLog.consumer.pspng_attributes.userSearchBaseDn = cn=users,dc=example,dc=edu
changeLog.consumer.pspng_attributes.userSearchFilter = uid=${}
  • No labels