This document is a work in progress. The wiki formatting could use improvement, your suggestions are welcome.
Getting Started with Real-Time Provisioning
Real-time provisioning is the incremental provisioning of groups, stems, and memberships triggered from the Grouper change log. Incremental provisioning is distinguished from full provisioning in that only a single or subset of an attribute's values are provisioned.
Real-time provisioning is available from the provisioning service provider (psp, formerly known as ldappcng) as of version 2.1.0. It should be possible to install and configure the psp for Grouper versions 1.6 and up.
These instructions assume that Grouper subjects are already provisioned to your ldap directory, and makes use of the vt-ldap based ldap source adapter.
These instructions were written on a Mac, other platforms should be similar.
Requirements
- Grouper API (requires Java 6 and a database, details are here)
- provisioning service provider (psp)
- ldap directory
Install : Grouper Installer
To install Grouper including the API, UI, WS, grouperClient, psp, etc., download and run the Grouper Installer.
As of version 2.1.0, the psp may be installed but not configured via the Grouper Installer.
curl
http://www.internet2.edu/grouper/release/2.1.0/grouperInstaller.jar
-O
java -jar grouperInstaller.jar
Install : Manual
Install the provisioning service provider by copying jar files and example configuration files from the psp distribution to your Grouper API installation.
Many jars are dependencies of the Shibboleth attribute resolver and may not be necessary in your deployment.
To install manually, download and unpack the psp, then copy jars and configuration files to your Grouper API installation.
Download and Unpack the PSP
Download the PSP here and unpack.
The distribution name is of the form grouper.psp-2.1.0.tar.gz
curl
http://www.internet2.edu/grouper/release/2.1.0/grouper.psp-2.1.0.tar.gz
-O
tar xzf grouper.psp-2.1.0.tar.gz
Copy Jars
Copy jars located in lib/custom
from the psp distribution to the Grouper API installation.
cp -vR grouper.psp-2.1.0/lib/custom/ grouper.apiBinary-2.1.0/lib/custom/
Copy Example Configuration Files
Copy example configuration files located in conf
from the psp distribution to the Grouper API installation.
Example psp configuration files are in directories named with the prefix psp-example-*
.
cp -vR grouper.psp-2.1.0/conf/ grouper.apiBinary-2.1.0/conf/
Configuration Example : Grouper to LDAP
This configuration example should apply to any ldap directory server.
examples |
|
DN structure |
bushy |
|
member DNs |
Configuration Example : Grouper to Tivoli
This configuration example targets an IBM Tivoli Directory Server with requirements from Penn State.
examples |
|
DN structure |
flat |
|
member subject ids |
|
group DNs |
|
member names |
|
group names |
Configuration Example : Grouper to OpenLDAP
This configuration example applies to OpenLDAP.
examples |
|
DN structure |
bushy |
|
member DNs |
|
member names |
|
group names |
Configuration Example : Grouper to Active Directory
Configuration Example : LDAP to Grouper
Configuration : SPMLv2
Provisioned Objects
Provisioned objects, in SPMLv2 terms, consist of identifiers, attributes (probably), and references (maybe) to the identifiers of other objects, which are most likely located on the same provisioning target.
Identifiers
Identifiers consist of a string ID, a target ID, and possibly a container ID. We consider a container ID to be similar to an ldap base dn. A container ID is itself an identifier, recursing potentially indefinitely.
Attributes
Name value pairs. Probably multi-valued. Case sensitive names and values. We return values in the same order as they were given to us.
References
A reference refers to the identifier of another object. It consists of two identifiers, the "from object" and the "to object". A node in a directed graph. Directional.
Configuration : Grouper
Configure Grouper Stem to be Provisioned
Configure the name of the Grouper stem to be provisioned, by default this is the root stem, which is the empty string.
The Grouper stem to be provisioned is configured in ldap.properties
.
# The base Grouper stem to be provisioned. edu.internet2.middleware.psp.baseStem=
Configure LDAP Provisioning Target
The LDAP provisioning target connection is configured in ldap.properties
.
Configure the default search base DN to match your directory :
edu.vt.middleware.ldap.baseDn = dc=example,dc=edu
Configure authentication and encryption :
edu.vt.middleware.ldap.serviceUser=cn=Manager,dc=example,dc=edu edu.vt.middleware.ldap.serviceCredential=secret
Configure the default base DN (container) for people and groups :
# The base DN for groups. edu.internet2.middleware.psp.groupsBaseDn = ou=groups,dc=example,dc=edu # The base DN for people. edu.internet2.middleware.psp.peopleBaseDn = ou=people,dc=example,dc=edu
Configure LDAP Subject Source
Configure Grouper to look for subjects in your LDAP directory by using the ldap source adapter.
The Grouper LDAP subject source connection is defined in sources.xml
.
<source adapterClass="edu.internet2.middleware.subject.provider.LdapSourceAdapter"> <id>ldap</id> <name>LdapSourceAdapter</name> <type>person</type> <init-param> <param-name>ldapProperties_file</param-name> <param-value>ldap.properties</param-value> </init-param>
Configure the base DN to match your directory in the various search configuration elements :
<param-name>base</param-name> <param-value>ou=people,dc=example,dc=edu</param-value> ...
Configure LDAP Subject Source in Grouper UI
Copy sources.xml
and ldap.properties
from the Grouper API to the Grouper UI.
cp grouper.apiBinary-2.1.0/conf/ldap.properties grouper.ui-2.1.0/dist/grouper/WEB-INF/classes/
cp grouper.apiBinary-2.1.0/conf/sources.xml grouper.ui-2.1.0/dist/grouper/WEB-INF/classes/
Copy vt-ldap.jar
from the Grouper API to the Grouper UI :
cp grouper.apiBinary-2.1.0/lib/custom/vt-ldap-3.3.4.jar grouper.ui-2.1.0/dist/grouper/WEB-INF/lib
The Grouper UI will not load unless you edit ldap.properties
in your Grouper UI installation or copy psp-ldap-target-2.1.0-SNAPSHOT.jar
to your Grouper UI installation.
Comment out or remove the psp specific search result handlers in ldap.properties
in your Grouper UI installation :
# edu.vt.middleware.ldap.searchResultHandlers=edu.internet2.middleware.psp.ldap.QuotedDnResultHandler,...
Configure LDAP Provisioning Structure
The LDAP structure and corresponding group RDN source attribute ID are configurable via macro replacement in ldap.properties
.
Flat
In a flat
structure all groups are provisioned under a single base DN (container ID). A flat
group's ldap RDN is its Grouper name or displayName.
Configure the flat
LDAP structure and name
group RDN source attribute ID in ldap.properties
:
edu.internet2.middleware.psp.structure=flat edu.internet2.middleware.psp.cnSourceAttributeID=name
Bushy
In a bushy
structure groups are provisioned hierarchically, with stems as branches (ldap organizationalUnits) in the tree. A bushy
group's RDN is its Grouper extension or displayExtension.
Configure the bushy
LDAP structure and extension
group RDN source attribute ID in ldap.properties
:
edu.internet2.middleware.psp.structure=bushy edu.internet2.middleware.psp.cnSourceAttributeID=extension
Configure Grouper Change Log
The Grouper change log is configured in grouper-loader.properties
.
To enable change log provisioning :
changeLog.consumer.psp.class = edu.internet2.middleware.psp.grouper.PspChangeLogConsumer
To schedule when the change log is processed :
changeLog.consumer.psp.quartzCron = 0 * * * * ?
To run full synchronizations periodically (by default every day at 5am) :
changeLog.psp.fullSync.class = edu.internet2.middleware.psp.grouper.PspChangeLogConsumer changeLog.psp.fullSync.quartzCron = 0 0 5 * * ?
To run a full synchronization job at loader startup :
changeLog.psp.fullSync.runAtStartup = true
The Quartz cron string documentation is here.
Configure Subject API Cache
The Subject API cache is configured in grouper.ehcache.xml
.
Adjust maxElementsInMemory
to be greater than or equal to the number of subjects.
Adjust timeToIdleSeconds
and timeToLiveSeconds
... accordingly ... ?
As of Grouper version 2.1.0, which uses Ehcache 2.4, statistics
must be "true"
to collect statistics which are logged at DEBUG
level.
<!-- Subject resolving caching --> <!-- @see CachingResolver#find(...) --> <cache name="edu.internet2.middleware.grouper.subj.CachingResolver.Find" maxElementsInMemory="5000" eternal="false" timeToIdleSeconds="30" timeToLiveSeconds="120" overflowToDisk="false" statistics="true" /> <!-- @see CachingResolver#findAll(...) --> <cache name="edu.internet2.middleware.grouper.subj.CachingResolver.FindAll" maxElementsInMemory="5000" eternal="false" timeToIdleSeconds="30" timeToLiveSeconds="120" overflowToDisk="false" statistics="true" /> <!-- @see CachingResolver#findByIdentifier(...) --> <cache name="edu.internet2.middleware.grouper.subj.CachingResolver.FindByIdentifier" maxElementsInMemory="5000" eternal="false" timeToIdleSeconds="30" timeToLiveSeconds="120" overflowToDisk="false" statistics="true" /> <!-- @see CachingResolver#findByIdOrIdentifier(...) --> <cache name="edu.internet2.middleware.grouper.subj.CachingResolver.FindByIdOrIdentifier" maxElementsInMemory="5000" eternal="false" timeToIdleSeconds="30" timeToLiveSeconds="120" overflowToDisk="false" statistics="true" />
Configure Grouper Logging
You may want to change the Grouper log file appenders in grouper.apiBinary-2.1.0/conf/log4j.properties
.
log4j.appender.grouper_error = org.apache.log4j.DailyRollingFileAppender log4j.appender.grouper_error.File = ${grouper.home}logs/grouper_error.log log4j.appender.grouper_error.DatePattern = '.'yyyy-MM-dd log4j.appender.grouper_event = org.apache.log4j.DailyRollingFileAppender log4j.appender.grouper_event.File = ${grouper.home}logs/grouper_event.log log4j.appender.grouper_event.DatePattern = '.'yyyy-MM-dd
Grouper Versions Prior to 2.1.0
The following changes are necessary to support provisioning Grouper API versions prior to 2.1.0 with the psp.
1. For versions prior to 2.1.0, there is a bug which will throw a NullPointerException if the following is not present in sources.xml
:
<search> <searchType>searchSubjectByIdentifierAttributes</searchType> <param> <param-name>filter</param-name> <param-value> (&(uid=%TERM%)(objectclass=person)) </param-value> </param> <param> <param-name>scope</param-name> <param-value>SUBTREE_SCOPE</param-value> </param> <param> <param-name>base</param-name> <param-value>ou=people,dc=example,dc=edu</param-value> </param> </search>
2. For versions prior to 2.1.0, the location of ldap.properties
specified in sources.xml
must be an absolute path. For versions 2.1.0 or later, the location of ldap.properties
may be an absolute path or in your Java classpath.
For example, Grouper API version 2.0.3 requires an absolute path to ldap.properties in sources.xml
:
<init-param> <param-name>ldapProperties_file</param-name> <param-value>/opt/grouper/2.0.3/grouper.apiBinary-2.0.3/conf/ldap.properties</param-value> </init-param>
For Grouper UI versions prior to 2.1.0, the path to ldap.properties
specified in sources.xml
will be different than in the Grouper API since the psp specific search result handlers must be commented out or removed in the Grouper UI :
<init-param> <param-name>ldapProperties_file</param-name> <param-value>/opt/grouper/2.0.3/grouper.ui-2.0.3/dist/grouper/WEB-INF/classes/ldap.properties</param-value> </init-param>
3. For Grouper API versions prior to 2.1.0, the ldap source adapter in subject.jar
does not provide the method which allows the psp to re-use the same ldap connection as the subject source. You will need to copy lib/grouper/subject.jar
from the Grouper 2.1.0 API distribution to your pre-2.1.0 Grouper API installation.
Configure : Provisioning Service Provider
The psp configuration files are :
|
Configuration for the objects, identifiers, attributes, and references to be provisioned to a target. |
|
Configuration for the Shibboleth attribute resolver. |
|
Configuration for Shibboleth services such as the attribute resolver, psp, and provisioning targets. |
|
Bootstraps Shibboleth. |
Configure PSP : Provisioned Objects, Identifiers, Attributes and References
The objects, identifiers, attributes, and references to be provisioned are defined in psp.xml
.
Configure PSP : Attribute Resolver
The values of the identifiers, attributes, and references to be provisioned are defined by a Shibboleth attribute resolver configuration psp-resolver.xml
.
Configure PSP : Logging and Output
The psp uses slf4j, and with Grouper, log4j configured in log4j.properties
.
# Provisioning : PSP (version 2.1+) log4j.logger.edu.internet2.middleware.psp = INFO # Provisioning : vt-ldap log4j.logger.edu.vt.middleware.ldap = INFO # Provisioning : ldap target # log4j.logger.edu.internet2.middleware.ldap = DEBUG # Provisioning : Grouper plugin to Shibboleth attribute resolver log4j.logger.edu.internet2.middleware.grouper.shibboleth = INFO
The psp is a Shibboleth service which is configured in psp-services.xml
.
property |
default |
value |
---|---|---|
|
true |
If |
|
false |
If |
|
false |
If |
|
<stdout> |
The path to the file to which SPML requests and responses are written. |
<!-- The provisioning service provider. --> <Service id="psp" xsi:type="psp:ProvisioningServiceProvider" depends-on="psp.AttributeAuthority" authority="psp.AttributeAuthority" logSpml="true" writeRequests="false" writeResponses="false" pathToOutputFile=""> <ConfigurationResource file="/psp.xml" xsi:type="resource:ClasspathResource"> <ResourceFilter xsi:type="grouper:ClasspathPropertyReplacement" xmlns="urn:mace:shibboleth:2.0:resource" propertyFile="/ldap.properties" /> </ConfigurationResource> </Service>
Configure PSP : LDAP Target
The LDAP target to be provisioned is a Shibboleth service configured in psp-services.xml
.
By default, the LDAP target to be provisioned re-uses the same vt-ldap
connection as the Grouper LDAP source adapter.
property |
default |
value |
---|---|---|
|
|
If |
|
|
If |
|
|
If |
|
|
The path to the file to which SPML requests and responses are written. |
|
|
Re-use the |
<!-- The ldap target. The ldapPoolIdSource is either "grouper" or "spring". --> <!-- If ldapPoolIdSource is "spring", the ldapPoolId must be the id of the ldap pool bean in the vt-ldap xml spring configuration. --> <!-- If ldapPoolIdSource is "grouper", the ldapPoolId must be the id of the LdapSourceAdapter in sources.xml --> <Service id="ldap" xsi:type="psp-ldap-target:LdapTarget" logSpml="true" ldapPoolId="ldap" ldapPoolIdSource="grouper"> <!-- A <ConfigurationResource/> is required to instantiate the <Service/>, so supply a do-nothing resource. --> <ConfigurationResource file="/edu/internet2/middleware/psp/util/empty-bean.xml" xsi:type="resource:ClasspathResource" /> </Service>
Provision Grouper
Before you can provision anything from Grouper to ldap or anywhere else, you will need to create the corresponding objects in Grouper using the UI, API, GSH, WS, loader, import, etc.
Provision Grouper : GSH
To calculate how a group should be provisioned :
bin/gsh.sh -psp -calc edu:group
To diff the current and correct provisioning of a group :
bin/gsh.sh -psp -diff edu:group
To provision or synchronize a group :
bin/gsh.sh -psp -sync edu:group
Provision Grouper : Grouper Change Log
To provision in real-time triggered by the Grouper change log, enable the psp consumer in grouper-loader.properties
and run the loader via
bin/gsh.sh -loader
Real-Time Provisioning Beta-Testing
Grouper Subject Sources
Institution |
Subject Source |
Number of Subjects |
Subject ID |
---|---|---|---|
LIGO |
LDAP |
1,000 |
dn: employeeNumber=882,ou=people,dc=ligo,dc=org |
Penn State |
LDAP |
165,000 |
dn:uid=xyx123,dc=psu,dc=edu |
UCLA |
LDAP |
40,000 |
|
UMontreal |
LDAP |
120,000 |
sAMAccountName (value same as cn) |
UVienna |
Undecided |
155,000 |
cn, uid |
UWMadison |
|
|
|
Provisioning Targets
Institution |
Target |
Implementation |
---|---|---|
LIGO |
LDAP |
OpenLDAP 2.4.x |
Penn State |
LDAP |
IBM Tivoli Directory Server |
UCLA |
LDAP |
Sun Java System Directory Server Enterprise Edition 6.3.1 |
UMontreal |
LDAP |
Active Directory |
UVienna |
LDAP |
Active Directory, OpenLDAP |
UWMadison |
|
|
Provisioning memberOf
The groups that a member is a member of may be provisioned to the memberOf attribute. Some LDAP implementations, such as Active Directory, automatically maintain the memberOf attribute. OpenLDAP maintains the memberOf attribute automatically via the memberOf overlay. The value of the memberOf attribute is typically a group DN.
Institution |
memberOf for members (people) |
memberOf for groups |
---|---|---|
LIGO |
+ |
|
Penn State |
+ |
|
UCLA |
- |
|
UMontreal |
automatic (Active Directory) |
automatic (Active Directory) |
UVienna |
automatic (Active Directory), OpenLDAP+memberOf |
automatic (Active Directory), OpenLDAP+memberOf |
UWMadison |
+ |
|
Provisioning eduMember
The eduMember objectClass defines the isMemberOf and hasMember attributes, whose values are identifiers which are not DNs.
Institution |
isMemberOf |
hasMember |
---|---|---|
LIGO |
+ |
+ |
Penn State |
+ |
+ |
UCLA |
uclaIsMemberOf |
uclaHasMember |
UMontreal |
- |
- |
UVienna |
- |
- |
UWMadison |
+ |
+ |
Provisioning eduCourse
The eduCourse objectClass defines course related attributes.
Institution |
eduCourse |
---|---|
LIGO |
- |
Penn State |
- |
UCLA |
- |
UMontreal |
- |
UVienna |
- |
UWMadison |
+ |
Provisioning Structure
The group provisioning structure may be either flat
or bushy
. A flat
structure provisions all groups into a single container. A bushy
structure provisions groups hierarchically.
For example, the DN of a group with name 'edu:stem:group' in a flat
structure looks like :
dn: cn=edu:stem:group,ou=groups,dc=example,dc=edu
while the DN of a group with name 'edu:stem:group' in a bushy
structure looks like :
dn: cn=group,ou=stem,ou=edu,ou=groups,dc=example,dc=edu
Institution |
Structure (flat or bushy) |
---|---|
LIGO |
bushy |
Penn State |
flat |
UCLA |
flat |
UMontreal |
bushy |
UVienna |
? |
UWMadison |
flat |
Membership Structure
Given groupA with memberA and groupB with memberB :
dn : cn=groupA,ou=groups member: cn=memberA,ou=people dn: cn=groupB,ou=groups member: cn=memberB,ou=people
If groupB is added as a member to groupA, how do you want groupA to be provisioned :
everything :
dn : cn=groupA,ou=groups member: cn=memberA,ou=people member: cn=memberB,ou=people member: cn=groupB,ou=people
immediate :
dn : cn=groupA,ou=groups member: cn=memberA,ou=people member: cn=groupB,ou=people
The everything membership structure handles applications which may not support nested groups and represents the nested structure of the group memberships.
The same membership structure applies to memberOf :
everything :
dn: cn=memberB,ou=people memberOf: cn=groupB,ou=groups memberOf: cn=groupA,ou=groups
immediate :
dn: cn=memberB,ou=people memberOf: cn=groupB,ou=groups
Institution |
member |
memberOf |
---|---|---|
LIGO |
everything |
everything |
Penn State |
|
|
UCLA |
|
|
UMontreal |
immediate |
immediate |
UVienna |
everything |
everything |
UWMadison |
|
|