Versions Compared

Old Version 33

changes.mady.by.user Bert Bee-Lindgren (gatech.edu)

Saved on

compared with

New Version 34

changes.mady.by.user Imholz

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Tried to clear up LDAP pool – LDAP connection.

...

Provisioning's job is to reflect Groups and their Memberships in other systems. This is handled in Grouper via the PSPNG component, which is mostly a client of the Grouper changelog and is designed as a ChangeLog Consumer (CLC). Detected changes in Grouper via the changelog are picked by PSPNG and evaluated for provisioning operations selectively.

 


Table of Contents

History

Over the years, dozens of provisioners have been created -- some focused on a single destination type and others with some generic functionality combined with a very wide variety of options and capabilities.

...

PSPNG's Configuration is done via the conf/grouper-loader.properties file, in the grouper API Binary, with a paragraph for each provisioning destination, as well as an additional paragraph that enables and configures FullSync operation. There are several configuration options documented in the following spreadsheet: 


 
Provisioner TypeParameterDefaultDescriptionDefault Behavior
All Provisioners
provisionerName
<required>
   



groupSelectionExpression
${
  utils.containedWithin(provisionerName,
     stemAttributes['etc:pspng:provision_to'], 
     groupAttributes['etc:pspng:provision_to'])
  &&  
  !utils.containedWithin(provisionerName, 
     stemAttributes['etc:pspng:do_not_provision_to'],
     groupAttributes['etc:pspng:do_not_provision_to'])
}
Jexl expression that refers to stem_attributes, group_attributes, or groupProvision groups if <provisionerName> is in a group or stem provision_to attribute AND NOT in a do_not_provision_to attribute

 
grouperDataCacheTime_secs
600 (seconds)
How long should Grouper (Group, Stem, Subject) data be cached by the provisioners?Grouper data will be cached for 10 minutes, though it is flushed when groups change.

 
grouperGroupCacheSize
10000 (Groups)
How many Grouper Groups should be kept in memory at a time? 

 
grouperSubjectCacheSize
10000 (Subjects)
How many Grouper Subjects should be kept in memory at a time? 

 
needsTargetSystemUsers
FALSE
Does provisioniner need User/Subject information from the Target System? For example, do any JEXL expressions need information that is not available in Grouper Subjects?All provisioning will be done based (only) on Grouper-Subject information. 

needsTargetSystemGroups
FALSE
Does provisioner need group information from the Target System? For example, this information could be used in various JEXL expressions.All provisioning will need to be done based on Grouper-Group information 

createMissingUsers
FALSE
Only used when needsTargetSystemUsers=true: Should users be created when they cannot be found?Users will not be created by Grouper Provisioning. Provisioning actions that require the users will fail.

 
userSearch_batchSize
50
Only used when needsTargetSystemUsers=true: How many users can be sought in a single Fetch?Fetches will seek information for up to 50 users at a single time. 

groupSearch_batchSize
50
Only used when needsTargetSystemGroups=true: How many groups can be sought in a single Fetch?Fetches will seek information for up to 50 groups at a time.

 
supportsEmptyGroups
TRUE
Can groups be created without any members? If so, it is easier to create them separately from membership changes.Yes, create groups as soon as possible.

 
sleepTimeAfterError_ms
1000
FullSync: Wait a bit before retrying a group that has failed. This prevents aggressive infinite loops.1 second pause before retrying a failed group.
LdapProvisioner (Abstract)
ldapPoolName
<required>
What ldap pool should be used by this provisioner 

 
userSearchBaseDn
null
Where to find users?Required if provisioner needsTargetSystemUsers=true 

userSearchFilter
null
Jexl expression that refers to stem_attributes, group_attributes, or groupHow to find users in the Target System? 

userSearchAttributes[]
dn,cn,uid,mail,
samAccountName,uidNumber,
objectclass
Comma-separated list of attributes that are useful for logging and that are needed by userSearchFilter or by a subclass's ValueFormatsReads common attributes from either Unix or ActiveDirectory LDAP servers

searchResultPagingEnabled
TRUE
Whether paging should be enabled in LDAP search requests; This typically requires the paging extension to be enabled and configured in LDAP to avoid LDAP Error Code 12.TRUE

 
ldapSearchResultPagingSize
100
How many result objects can be pulled by a single request. This is small to avoid problems by default.Break the results of a large query into fairly tiny chunks.

 
ldapUserCacheTime_secs
600
How long to keep User information in memory?Keep User information in memory for 10 minutes, though user-information is flushed when users are changed by a provisioner 

ldapUserCacheSize
10000
How many LDAP accounts can be kept in memory at a time, indexed by the Subject mapped to them?Keep the last 10000 users found by searching with Subject information 

isActiveDirectory
FALSE
Is this an active-directory server? If so, then AD-specific attribute-value-paging is enabled. Also, member (reverse user-to-group virtual attribute) is enabled.LDAP server is treated like a non-active-directory server. Problems will occur with full-sync of large groups.

 
maxValuesToChangePerOperation
100
How many values can be added/removed from an attribute in a single ldap operationBreaks large list of values that need to be added/removed from an attribute into chunks that this size. For example, 5000 values that need to be added would be added in 50 chunks of 100 values each.
LdapGroupProvisioner (Also Provisioner and LDAPProvisioner)
memberAttributeName
'member' for AD
<required> otherwise
What attribute represents a group's members in the Target System?Active Directory should just work. Otherwise, this is required. 

memberAttributeValueFormat
${ldapUser.dn}
What value (typically based on Subject or TargetSystemUser information) is written into the memberAttributeName attribute of groups?
Active Directory and GroupOfUniqueNames will typically work. This is a JEXL expression and is parsed at runtime. You may choose to script your way into this, by perhaps choosing a specific attribute:
${ldapUser.getStringValue("uid")}

 
groupAttributeName
memberof for AD
null otherwise
Virtual attribute of accounts that lists their groups 

 
groupCreationLdifTemplate
null
What LDIF should be written to the directory to add a group. Multiple lines need to be separated by || (double-pipes). The DN of the LDIF will be combined with groupCreationBaseDn>  

groupCreationBaseDn
<groupSearchBaseDn>
Where should groups be created? At group-creation time, this is appended to the DN that results from the groupCreationLdifTemplate.Groups are created starting at the top of the search BaseDn. 

groupSearchBaseDn
<required>
Where are groups found? 

 
grouperIsAuthoritative
FALSE
Should groups in the groupSearchBaseDn/allGroupSearchFilter be removed if they no longer exist in Grouper? 

 
allGroupSearchFilter
null
FUTURE: How to find all the groups that grouper-provisioning maintains. If <grouperIsAuthoritative>, then groups found via this filter will be removed during a full sync.Groups are not removed when they are removed from Grouper nor when they no longer match the groupSelectionExpression. 

singleGroupSearchFilter
<required>
How to find a group, based on Grouper Group (or stem) information  

groupSearchAttributes
cn,gidNumber,samAccountName,objectclass
Attributes that should be read from groups when searching for them. This needs to include all the attributes used in singleGroupSearchFilter. This should not include the attribute which holds the group's members.Support common, basic singleGroupSearchFilters.

 
ldapGroupCacheTime_secs
600
How long should LDAP-Group information be cached in memory?Keep LDAP Group information in memory for 10 minutes, though it is flushed when users are changed by a provisioner.

 
ldapGroupCacheSize
10000
How many LDAP groups to keep in memory, indexed by Grouper Group. 

 
needsTargetSystemUsers
TRUE
See above (JEXL expressions use User and Group information from the Target System)  

needsTargetSystemGroups
TRUE
See above (JEXL expressions use User and Group information from the Target System) 
LdapAttributeProvisioner 
 (Also Provisioner and LDAPProvisioner)
provisionedAttributeName
<required>
What attribute is changed in User LDAP objects to represent group membership? 

 
provisionedAttributeValueFormat
${group.name}
What value (typically based on Subject or TargetSystemUser information) is written into the provisionedAttributeName users?The stem:Group name is written to the attribute specified in <provisionedAttributeName> 

needsTargetSystemUsers
TRUE
See above (JEXL expressions only use User information from the Target System)  

needsTargetSystemGroups
FALSE
See above (JEXL expressions only use User information from the Target System) 

 
allProvisionedAttributePrefix
null
What values of the attribute is grouper authoritative for during a full sync? null (default) or empty means that pspng will only process removals as memberships change, and won't clean up unknown attribute values.Warning: Grouper should have full control over the target attribute to avoid complications that come from sharing attributes with multiple provisioning tools.
LDAP Properties
 

LDAP configuration is done based on the ldaptive library's property configuration. A paragraph of ldap configuration is created in grouper-loader.properties for each LDAP endpoint, and that paragraph is referenced by the appropriate provisioners.   Comments in grouper-loader.base.properties refers to these "LDAP pools" as "LDAP connections".

 Warning
At this time, the LDAP bindCredential cannot be encrypted via the Grouper morphstring.
  
ldap.groupOfNames.ldapUrl = ldaps: //hostname
ldap.groupOfNames.bindDn = cn=xxxxxx,ou=xxxxx
ldap.groupOfNames.bindCredential = xxxxx
ldap.groupOfNames.someOtherLdapProperty = value
  
# The LDAP block is connected to each provisioner via

# in another paragraph a provisioner is associated with the LDAP pool name
changeLog.consumer.pspng_groupOfUniqueNames.ldapPoolName = groupOfNames 

Advanced Ldaptive Properties
PSPNG relies on the Ldaptive library for all LDAP-related operations. To learn more about what other LDAP properties are available,  one simple example can be found here. Moving into more realistic examples will probably be helped by looking at the ldaptive configuration classes and the setters available within them: connections, pooling, binding (sasl, gssapi, x509, jks, etc).
In case it is helpful, this is currently implemented here. You may also wish to take a look at GRP-1306 to learn more about the differences between vtdap (the previously used LDAP library) vs ldaptive.
Run PSPNG
PSPNG runs as part of the grouper loader. So simply run the loader with:
 Code Block
cd <grouper-api-binary-folder>/bin
./gsh -loader
 

Group/Folder Selection
Groups can be enabled and disabled based on Group or Folder/Stem information and attributes. By default, two multivalued attributes are used:
...
These attributes need to be assigned to Groups or Folders via the Lite UI. 
 

After the attribute is assigned to a folder or group, you need to assign it a value. The value of the attribute  MUST  match the provisioner name that is defined in the grouper-loader.properties configuration (i.e. pspng_groupOfUniqueNames).
...
To troubleshoot PSPNG configuration, you may need to add the following logging configuration to the log4j.properties file:
 

 Code Block
languagebash
log4j.logger.edu.internet2.middleware.grouper.pspng=DEBUG
log4j.logger.edu.internet2.middleware.grouper.changeLog=DEBUG
...
  1. Grouper/PSPNG has no ability to keep accounts up to date; it only creates them if they're missing, and
  2. Your Grouper-subject mappings probably don't have some of the information needed for account provisioning. And, even if all you need is name, username, and email address (which are probably in your subject mappings), you'll still run into problem (1) where Grouper offers no mechanism to update name and email address when they change in your subject source.
...

Full SYNC Provisioning
 

 Warning
titleUpgrade warning (PSPNG Patch #11, June 8, 2017)
The Full-Sync-provisioning improvements in Grouper 2.3/PSPNG Patch #11 are configured with new, better properties. (You should no longer use the properties that start with changeLog.psp.fullSync.)  

In the background of PSPNG, there is always a full-sync-provisioning engine running which is automatically used when incremental provisioning finds conflicting changes or otherwise is unable to handle the changelog events. The full-sync items in grouper-loader.properties do not alter/configure the background engine but instead define quartz jobs that send all the groups marked for provisioning into the queues that drive the full-sync engine. Additionally, if a provisioner is "authoritative" (changeLog.consumer.<provisioner>.grouperIsAuthoritative=true), then a cleanup task is included in the scheduled job which will delete extra groups or attributes that no longer exist in the Group Registry. (Note: Data in a provisioning target will always get deleted by  incremental  provisioning when groups are deleted, regardless whether a provisioner is authoritative. Full-sync cleanup is just a safety net in case incremental provisioning misses something, or if the data was written into the target system outside of PSPNG.)
...
 Code Block
languagebash
themeEclipse
changeLog.consumer.pspng_groupOfUniqueNames.class = edu.internet2.middleware.grouper.pspng.PspChangelogConsumerShim
changeLog.consumer.pspng_groupOfUniqueNames.type = edu.internet2.middleware.grouper.pspng.LdapGroupProvisioner
changeLog.consumer.pspng_groupOfUniqueNames.quartzCron = 0 * * * * ?
changeLog.consumer.pspng_groupOfUniqueNames.ldapPoolName = opendj
changeLog.consumer.pspng_groupOfUniqueNames.memberAttributeName = uniqueMember
# changeLog.consumer.pspng_posixGroup.memberAttributeValueFormat = ${ldapUser.getStringValue("uid")}
changeLog.consumer.pspng_groupOfUniqueNames.memberAttributeValueFormat = ${ldapUser.getDn()}
changeLog.consumer.pspng_groupOfUniqueNames.groupSearchBaseDn = ou=grouper,ou=groups,dc=example,dc=edu
changeLog.consumer.pspng_groupOfUniqueNames.allGroupsSearchFilter = objectclass=groupOfUniqueNames
changeLog.consumer.pspng_groupOfUniqueNames.singleGroupSearchFilter = (&(objectclass=groupOfUniqueNames)(cn=${group.name}))
changeLog.consumer.pspng_groupOfUniqueNames.groupSearchAttributes=cn,gidNumber,objectclass
changeLog.consumer.pspng_groupOfUniqueNames.groupCreationLdifTemplate = dn: cn=${group.name}||cn: ${group.name}||objectclass: groupOfUniqueNames
changeLog.consumer.pspng_groupOfUniqueNames.userSearchBaseDn = cn=users,dc=example,dc=edu
changeLog.consumer.pspng_groupOfUniqueNames.userSearchFilter = uid=${subject.id}
...

POSIX GROUPS
 Code Block
languagebash
changeLog.consumer.pspng_posixGroup.class = edu.internet2.middleware.grouper.pspng.PspChangelogConsumerShim
changeLog.consumer.pspng_posixGroup.type = edu.internet2.middleware.grouper.pspng.LdapGroupProvisioner
changeLog.consumer.pspng_posixGroup.quartzCron = 0 * * * * ?
changeLog.consumer.pspng_posixGroup.ldapPoolName = opendj
changeLog.consumer.pspng_posixGroup.memberAttributeName = memberUid
changeLog.consumer.pspng_posixGroup.memberAttributeValueFormat = ${ldapUser.getStringValue("uid")}
changeLog.consumer.pspng_posixGroup.groupSearchBaseDn = ou=grouper-posix,ou=groups,dc=example,dc=edu
changeLog.consumer.pspng_posixGroup.allGroupsSearchFilter = objectclass=posixGroup
changeLog.consumer.pspng_posixGroup.singleGroupSearchFilter = (&(objectclass=posixGrouper)(cn=${group.name}))
changeLog.consumer.pspng_posixGroup.groupSearchAttributes=cn,gidNumber,objectclass
# Obviously, gidNumber should be based on a grouper-group attribute
changeLog.consumer.psong_posixGroup.groupCreationLdifTemplate = dn: cn=posix-${group.name}||cn: posix-${group.name}||objectclass: posixGroup||objectclass: groupOfNames||gidNumber: ${group.idIndex}
changeLog.consumer.pspng_posixGroup.userSearchBaseDn = cn=users,dc=example,dc=edu
changeLog.consumer.pspng_posixGroup.userSearchFilter = uid=${subject.id}
...

ACTIVE DIRECTORY GROUPS
 Code Block
languagebash
changeLog.consumer.pspng_activedirectory.class = edu.internet2.middleware.grouper.pspng.PspChangelogConsumerShim
changeLog.consumer.pspng_activedirectory.type = edu.internet2.middleware.grouper.pspng.LdapGroupProvisioner
changeLog.consumer.pspng_activedirectory.quartzCron = 0 * * * * ?
changeLog.consumer.pspng_activedirectory.ldapPoolName = active_directory
changeLog.consumer.pspng_activedirectory.isActiveDirectory = true
changeLog.consumer.pspng_activedirectory.memberAttributeName = member
changeLog.consumer.pspng_activedirectory.memberAttributeValueFormat = ${ldapUser.getDn()}
changeLog.consumer.pspng_activedirectory.groupSearchBaseDn = ou=grouper,ou=groups,dc=example,dc=edu
changeLog.consumer.pspng_activedirectory.allGroupsSearchFilter = objectclass=group
changeLog.consumer.pspng_activedirectory.singleGroupSearchFilter = (&(objectclass=group)(cn=${group.name}))
changeLog.consumer.pspng_activedirectory.groupCreationLdifTemplate = dn: cn=${group.name}||cn: ${group.name}||objectclass: group
changeLog.consumer.pspng_activedirectory.userSearchBaseDn = cn=users,dc=example,dc=edu
changeLog.consumer.pspng_activedirectory.userSearchFilter = samAccountName=${subject.id}
...