has been upgraded to Confluence 6.5.0. If you have any questions and/or concerns, please contact us at


By Tom Barton, Internet2 and the University of Chicago

This year, the National Science Foundation’s proposal solicitation for its Campus Cyberinfrastructure (CC*) program has new InCommon-related requirements. These requirements help ensure that campus researchers can successfully use their campus credentials to access research related services available via global federation (InCommon and eduGain).

The solicitation says this about campus cyberinfrastructure plans:

The plan should include the campus status and plans with respect to federated identity and specifically InCommon, including: if the campus is registered with InCommon as supporting the Research and Scholarship (R&S) Entity Category to streamline integration with research applications; and if the campus meets the InCommon Baseline Expectations for Trust in Federation.

Meeting the Research & Scholarship Entity Category Requirement

What and Why

Many federated research services require a few user attributes to successfully complete login, usually name, email, and a persistent user identifier (called the “R&S attribute bundle”). A common example is CILogon, which enables a user’s campus credential to be used to access a variety of research infrastructures such as XSEDE.

A global program and standard for the R&E sector has been established to facilitate meeting this need to share these attributes. That’s the Research & Scholarship Entity Category (R&S). That standard enables research services to request that their national R&E federation (as InCommon is for the US) “tag” them with the R&S entity category and specifies the vetting that R&E federation operators perform to ensure that such a tag is appropriate. It also provides a means by which a campus federated login system (called an Identity Provider or IdP in federation lingo) can automatically release the R&S attribute bundle to research services that have been tagged R&S, and a corresponding R&S tag to be given to an IdP to signal that it participates in this global program. This is important because some research services will only permit a login to proceed if the user’s campus IdP is so tagged.

Here are links to the R&S Entity Category specification and an FAQ addressing both IdP operators and Service Provider (SP) operators.


Check to see if your IdP or SP is listed

as already meeting the “REFEDS R&S Entity Category specification” by being displayed as “research-and-scholarship” on a green background. Those shown a red background are a legacy from before there was global agreement on the R&S Entity Category. Their R&S attributes are limited to InCommon services only, hindering user access to international research services.

A wiki page describes how to enable an IdP to automatically provide the R&S attribute bundle to R&S tagged services. Once you have done so, email to request that your IdP be tagged as REFEDS R&S.

If a research service operated within the campus needs to support federated access for users elsewhere, fill out this application to be given the R&S tag.

Meeting the Baseline Expectations Requirement

What and Why

When service providers rely on federation, they are relying on those who operate IdPs to do a reasonable job of ensuring that their logins are trustworthy. In other words, that an account is used only by the user to whom it is assigned, and that the user is currently authorized to use it. Similarly, when IdPs send user attributes to an SP, the IdP operators rely on the service provider to use those attributes only for the stated purpose and to reasonably prevent unauthorized disclosure. Federation users rely on IdP and SP operators to operate their services in a manner that makes it as easy as possible for them to access the services they need.

The Baseline Expectations for Trust In Federation is a small set of high level statements that describe how InCommon participants expect InCommon IdPs and SPs to be operated to embody reasonable trust and usability, and how InCommon Federation operators should support this objective. The statements themselves as well as an extensive implementation program are detailed on the Baseline Expectations for Trust in Federation wiki page.


Step 1: Metadata review & update

There are several clearly defined requirements of an IdP’s or SP’s federation metadata that should be reviewed to ensure they meet Baseline Expectations. One or more campus IT people are InCommon Site Administrators, which authorizes them to manage campus IdP and SP metadata using InCommon’s Federation Manager. They should login, review entity metadata, and make any changes that may be needed. The most common updates that are needed or strongly recommended are supplying an institutional logo (for both IdPs and SPs) and an error URL (for IdPs). These are key enablers of good user experience. Other common needs are to provide security contact information and a privacy policy URL.

Step 2: Operational practice review

Good institutional judgment is used to address some of the other Baseline Expectations. Some guidance is provided here, but please note that further interpretation may occur as consensus of the InCommon participant community about these statements develops. If you are unsure whether your operations currently or will reasonably soon meet these expectations, please raise your question on the InCommon Participants mailing list.

For IdPs, the following Baseline Expectations express what constitutes a reasonable job of ensuring that IdP logins are trustworthy.

  1.  The IdP is operated with organizational-level authority

  2.  The IdP is trusted enough to be used to access the organization’s own systems

  3.  Generally-accepted security practices are applied to the IdP

The first statement holds when responsibility for operating the institution’s IdP is assigned appropriately, usually to the central IT organization. The second statement asserts that the quality of the IdP operation is equivalent to the quality of any other authentication service that is used to login to important systems, such as HR, course management, or grants administration systems. And the third one recognizes that user accounts are points of potential compromise, so reasonable security protections and security incident response measures should apply to IdP operations in a manner equivalent to how other authentication services are protected by the organization.

For SPs, the following Baseline Expectations express what constitutes a reasonable job of ensuring that attributes are used only for the stated purpose and are protected from unauthorized disclosure.

  1. Controls are in place to reasonably secure information and maintain user privacy

  2. Information received from IdPs is not shared with third parties without permission and is stored only when necessary for SP’s purpose

  3. Generally-accepted security practices are applied to the SP

As with the IdP statements, these express the expectation that an SP system that collects user data will protect it in alignment with how your organization determines protections for other systems that store or process personal information.

Note that there are at present no external standards covering statements 1-3 (for IdPs or for SPs) that all InCommon participants are required to meet. Basically, these statements amount to an “eat your own dogfood” expectation. If you don’t, why should others trust you?

InCommon is pleased to announce general availability of version 3.0 of the InCommon Federation Manager. Release notes for this version of the Federation Manager may be found on the wiki.

This release introduces responsive design (targeted for use on mobile devices) for the Site Administrator functions and provides Site Administrators with a new "dashboard" from which they can easily determine the status of their metadata, as well as add and modify metadata.  This simplifies the submission of metadata.  More simplification is planned for upcoming releases.  If you are interested in what features are targeted for future releases, please take a look at the Federation Manager Roadmap.

This roadmap and the Federation Manager releases this year were made possible by the InCommon dues increase enacted by InCommon steering in the fall of 2016.  One target for that dues increase was 'hardening of InCommon infrastructure', and this work is part of that commitment.

The InCommon metadata aggregate is approaching 43MB and continues to grow. To compensate, InCommon Operations will enable HTTP Compression on the metadata server ( as follows:

Friday, April 7: Preview Aggregate (InCommon-metadata-preview.xml)
Tuesday, April 11: IdP-only Aggregate (InCommon-metadata-idp-only.xml)

The remaining metadata aggregates (including the main production aggregate) will be enabled for HTTP Compression at a later date TBD.

There is nothing you need to do to realize the benefits of HTTP Compression. On or after the above date, if you are consuming either the Preview Aggregate or the IdP-only Aggregate, you may observe reduced bandwidth use and decreased download time during metadata refresh. This is especially true of the Preview Aggregate, which is just 8MB compressed.

The client and the server negotiate compression automatically. The server will send a compressed metadata aggregate if (and only if) your metadata client indicates support for compression in the HTTP request. Check your client software documentation for details.

If you have any concerns or experience any issues, please contact us at

A Progress Report on the Migration to Shibboleth IdP V3

You’ve probably heard by now that Shibboleth IdP V2 reached end-of-life on July 31, 2016. Going forward, no bug fixes—not even security-related bug fixes—will be issued by the Shibboleth Project. In other words, Shibboleth IdP V2 is unsupported software.

Shibboleth IdP operators in the InCommon federation and other federations around the world are actively Upgrading to Shibboleth IdP V3. This article summarizes the progress of that effort over the last six (6) months.

All statistics in this article were compiled on January 2, 2017.

Here are some interesting facts:

  • Nearly 90% of the IdPs registered by InCommon are Shibboleth IdPs

  • Three-fourths of the world’s IdP deployments are based on the Shibboleth software

  • Four federations deploy three-fourths of the world’s Shibboleth IdPs

  • The same four federations account for three-fourths of the Shibboleth IdP V3 deployments worldwide

  • More than half (23/40) of the world’s federations each deploy fewer than three (3) Shibboleth IdPs

  • Fifteen federations deploy no Shibboleth IdPs whatsoever

Read on for details!

Shibboleth IdPs in InCommon Metadata

Nearly 90% of the IdPs registered by InCommon are Shibboleth IdP deployments

Of the 451 IdPs registered by InCommon, 403 (89%) are Shibboleth IdPs. Of these, 226 (56%) are Shibboleth IdP V3 deployments.

 InCommon metadata
Number of Shibboleth IdPs403 (89%)
Number of non-Shibboleth IdPs48
Total number of IdPs451
 InCommon metadata
Number of Shibboleth IdP V3226 (56%)

Number of Shibboleth IdP V2

149 (37%)
Number of Shibboleth IdP V?28 (7%)
Total number of Shibboleth IdPs403

In the last six (6) months, the number of V3 IdPs registered by InCommon has increased from 125 to 226. As a percentage, V3 deployments increased from 32% to 56% over the same time period.

 InCommon metadata
InCommon metadata
Number of IdPs433451
Number of Shibboleth IdPs389 (90%)403 (89%)
Number of Shibboleth IdP V3125 (32%)226 (56%)

For more information: Lists of Shibboleth IdPs Registered by InCommon

Now let’s take a look at global migration activity by examining eduGAIN metadata.

Shibboleth IdPs in eduGAIN Metadata

Four federations deploy three-fourths of the world’s Shibboleth IdPs:


Of the 2262 IdPs in eduGAIN metadata, 1690 (75%) are Shibboleth IdPs. Of these, 818 (48%) are Shibboleth IdP V3 deployments.

 eduGAIN metadata
Number of Shibboleth IdPs1690 (75%)
Number of non-Shibboleth IdPs572
Total number of IdPs2262
 eduGAIN metadata
Number of Shibboleth IdP V3818 (48%)

Number of Shibboleth IdP V2

696 (41%)
Number of Shibboleth IdP V?176 (11%)
Total number of Shibboleth IdPs1690

In the last six (6) months, the number of V3 IdPs in eduGAIN metadata has increased from 397 to 818. As a percentage, V3 deployments have nearly doubled over the same time period, increasing from 25% to 48%.

 eduGAIN metadata
eduGAIN metadata
Number of IdPs21342262
Number of Shibboleth IdPs1603 (75%)1690 (75%)
Number of Shibboleth IdP V3397 (25%)818 (48%)

For a complete analysis, including the number of Shibboleth IdP V3 deployments per registrar, see: Summary of Shibboleth IdPs in eduGAIN Metadata

For More Information

For the latest news and progress, visit this wiki page, which is updated regularly:

All of the data in this article were compiled using the Shibboleth IdP Probe, a script that probes a sequence of Shibboleth IdPs and determines the version of the Shibboleth software used by each deployment. The software can be downloaded from GitHub.

New and Emerging Methods of SAML Metadata Distribution

In September 2016, the Per-Entity Metadata Working Group, led by Scott Koranda, submitted an interim report to the InCommon Technical Advisory Committee with the following recommendation:

The working group finds and recommends that InCommon Operations proceed immediately with the design, creation, and delivery of a new InCommon metadata aggregate that contains only the metadata for identity providers (IdPs). The new IdP-only aggregate will help relieve issues some SPs face as the size of the existing InCommon metadata aggregates continues to grow.

Subsequently, on October 4, InCommon Ops announced the general availability of a production-quality IdP-only metadata aggregate for SP deployments. See the IdP-only Aggregate wiki page for details.

Using the IdP-only metadata aggregate

The new IdP-only metadata aggregate is intended for use by SP deployments only. IdP deployments should continue to use the main production aggregate described on the Metadata Aggregates wiki page.

Characteristics of the IdP-only Aggregate

The IdP-only metadata aggregate is approximately 16MB, which is about 42% of the size of the full InCommon aggregate.

Determining the size of the IdP-only aggregate
$ curl --silent --head $MD_LOCATION1 | grep -F Content-Length
Content-Length: 38623782
$ curl --silent --head $MD_LOCATION2 | grep -F Content-Length
Content-Length: 16438778

As of October 10, the IdP-only metadata aggregate contains 2231 entity descriptors, of which 447 are registered by InCommon. Each entity descriptor contains an <md:IDPSSODescriptor> child element. Seven (7) of the entities contain an <md:SPSSODescriptor> child element as well.

Determining the number of entities in the IdP-only aggregate
# For a description of the output of the count_entity_roles.xsl script, see:
$ curl --silent $MD_LOCATION1 | xsltproc ./count_entity_roles.xsl -
$ curl --silent $MD_LOCATION2 | xsltproc ./count_entity_roles.xsl -

For a complete up-to-date list of IdPs in InCommon metadata, see the List of IdP Display Names wiki page.

Benefits and Risks of the IdP-only Aggregate

What is per-entity metadata?

The SAML specification defines two entities: the Identity Provider (a producer of SAML assertions) and the Service Provider (a consumer of SAML assertions). A Service Provider requires the “metadata” of the Identity Provider (and vice versa). The metadata describe a SAML deployment, providing security, privacy, and interoperability to the relying party.

As a practical matter, SAML metadata is batch distributed as an aggregate of entity descriptors. With the proliferation of global aggregation services such as eduGAIN, the size of aggregates has grown dramatically, which is causing federations to re-examine existing methods of metadata distribution.

The term “per-entity metadata” refers to a single entity descriptor. The Metadata Query Protocol is an emerging standard that describes how to obtain per-entity metadata from a trusted oracle. Since the entity descriptor is the basic unit of policy and interoperability, this method of metadata distribution is both logical and efficient.

Since the IdP-only metadata aggregate is significantly smaller than the full aggregate, the former buys valuable time for service provider deployments—especially modestly provisioned deploymentsuntil per-entity metadata becomes readily available. For one particular class of service providers, the IdP-only aggregate will continue to be essential infrastructure long after other InCommon deployments have migrated to per-entity metadata. The rest of this section explains why this is so.

The vast majority of SPs do not have a dynamic discovery interface (i.e., a discovery interface that depends on published metadata) and so these SPs will be able to leverage per-entity metadata without delay. In fact, many of these SPs depend on a small number of fixed IdPs so the migration to per-entity metadata will be straightforward for them.

On the other hand, for the relatively few SPs that implement a dynamic discovery interface, the benefit of per-entity metadata is less clear since these SPs currently require an aggregate for IdP discovery. We expect these SPs to consume the IdP-only aggregate until the community addresses the IdP discovery issue brought about by per-entity metadata.

Be aware that there is no fallback aggregate of IdP-only metadata. In that sense, there is some risk associated with the use of the IdP-only aggregate. If you must fall back, you will have no choice but to fall back to the full Fallback Aggregate described on the Metadata Aggregates wiki page.

The Future is Per-entity Metadata

The Per-Entity Metadata Working Group is expected to submit its final report to the InCommon TAC by November 2016, after the community has reviewed the report. We anticipate that the working group will recommend that InCommon Operations deploy a production-quality metadata query server and that all InCommon SAML deployments (except those SPs that implement a dynamic discovery interface as discussed above) migrate to per-entity metadata as soon as possible.

Eventually all SAML deployments will benefit from per-entity metadata. IdP deployers, in particular, are anxiously awaiting the arrival of a metadata query server, and we expect many IdPs will be among the first deployments to realize the benefits of per-entity metadata.

Two SAML implementations are known to support the Metadata Query Protocol: simpleSAMLphp and Shibboleth. (See the MDQ Client Software wiki page for more information.) In particular, support for the Metadata Query Protocol was introduced in version 3 of the Shibboleth IdP software. Shibboleth IdP deployments that have upgraded to Shibboleth IdP V3 will be among the first to migrate to per-entity metadata.

Shibboleth IdP V2 End-of-Life

Shibboleth IdP V2 reached end-of-life on July 31, 2016. In the future, no bug fixes, not even security-related bug fixes, will be issued. Upgrade to Shibboleth IdP V3 now!

Other SAML software will benefit from per-entity metadata as well. For example, Microsoft AD FS can be configured to retrieve a single entity descriptor from a metadata query server, which is a huge step in the right direction. The hope is that AD FS and other SAML implementations will eventually support the RESTful Metadata Query Protocol like simpleSAMLphp and Shibboleth.

Scoped User Identifiers

Recently a serious flaw was found in Office 365:

You should of course review the report and make your own determination but here’s a spoiler: The Office 365 application neglected to scope-check a user identifier, which allowed an arbitrary identity provider to assert any user identifier whatsoever and thereby gain unauthorized access to the application.

Here are a few lessons learned from the Office 365 vulnerability.

Lesson Learned #1

An email address is not a user identifier.

The Office 365 incident revolved around the following SAML attribute:

The IDPEmail attribute
<saml2:Attribute FriendlyName="IDPEmail"

From a SAML perspective, there are a number of problems with the IDPEmail attribute shown above but the point that matters most is: The IDPEmail attribute is not an email address at all. Its value may look like an email address, but in fact, the IDPEmail attribute corresponds to the User Principal Name (userPrincipalName) of the existing account of the user in Azure AD.

That’s an important observation: The IDPEmail attribute is an identifier, and moreover, it is actually used by the Office 365 application for access control. That leads directly to our next observation.

Lesson Learned #2

All user identifiers must be scope-checked by the relying party.

Some identifiers are explicitly scoped. In other cases, an identifier is implicitly scoped to the issuer. In all cases, a user identifier must be scope-checked by the relying party.

Scoped Attributes

Some user identifiers are explicitly scoped. For example, the so-called scoped attributes eduPersonPrincipalName and eduPersonUniqueId encode a “scope” directly in the attribute value:

Two scoped attributes
<saml2:Attribute FriendlyName="eduPersonPrincipalName"

<saml2:Attribute FriendlyName="eduPersonUniqueId"

In the above examples, the “scope” is “” and “,” respectively. Only one identity provider (IdP) is authorized to assert the scope in each case, namely, the Ohio State University IdP and the Internet2 IdP, respectively. A relying party must check the asserted scope just-in-time and be prepared to reject an identifier if it does not pass the scope check.

So how does the relying party know what “scope” an IdP is authorized to assert? Trusted metadata, of course! If you search the InCommon metadata aggregate you will find the following <shibmd:Scope> elements:

Scopes in metadata
<shibmd:Scope regexp="false"></shibmd:Scope>
<shibmd:Scope regexp="false"></shibmd:Scope>

Each <shibmd:Scope> element is associated with the IdP authorized to assert that scope, namely, the Ohio State University IdP and the Internet2 IdP, respectively.

Other Scoped Identifiers

The SAML2 Persistent NameID is explicitly scoped but in a completely different manner. Here is an example:

SAML2 Persistent NameID

The SAML2 Persistent NameID is asserted by the IdP as a child element of the <saml2:Subject> element or the eduPersonTargetedID attribute. In either case, the NameQualifier XML attribute must be scope-checked, that is, it must match the actual issuer of the assertion. Otherwise the identifier should be discarded by default.

All of the examples thus far have been SAML attributes and identifiers but scope-checking is not a SAML issue per se. For instance, the OpenID Connect "sub" claim is a locally unique, non-reassigned identifier for the user (sub = subject). Here is an example of an ID token asserted by an OpenID provider:

An OpenID token
  "iss": "",
  "sub": "24400320",
  "aud": "s6BhdRkqt3",
  "exp": 1311281970,
  "iat": 1311280970

By definition, the "sub" claim is implicitly scoped to the issuer. In practice, a relying party would store the pair (iss, sub) and subsequently scope-check an ID token by matching both the "iss" value and the "sub" value. This prevents an arbitrary OpenID provider from asserting an unauthorized "sub" value, intentionally or otherwise.

Other Identifiers

At this point we’ve looked at the following identifiers:

  • eduPersonPrincipalName

  • eduPersonUniqueId

  • SAML2 Persistent NameID

  • eduPersonTargetedID

  • OpenID Connect "sub" claim

These aren’t the only identifiers that need to be scope-checked, however. See the NameIdentifiers topic in the Shibboleth wiki for a more complete list.

All of the above identifiers have well-defined scope semantics. The IDPEmail attribute, OTOH, is ill-suited for cross-domain access control. While its name and value suggest an email address, the underlying identifier (userPrincipalName) has no documented scope semantics AFAIK.

Lesson Learned #3

If your federating software doesn’t scope-check user identifiers, then that responsibility must be taken up by the application software (i.e., the application developer).

Note well: The application developer must scope-check all identifiers asserted by untrusted 3rd parties. This is especially true if the identifier is used for access control. Failure to do so may lead to major security holes like the one reported in Office 365.

Of course this assumes the application relies on scoped identifiers to begin with. In particular, an application should never rely on an email address to identify a user. An email address is not scoped. For instance, the email address may be legitimately asserted by any IdP. Conclusion: an email address makes a lousy user identifier

Explicitly scoped identifiers (such as eduPersonPrincipalName, eduPersonUniqueId, and SAML2 Persistent NameID) are preferred since smart middleware can do the work for you. For instance, the Shibboleth SP software scope-checks the above identifiers by default. In the case of the scoped attributes eduPersonPrincipalName and eduPersonUniqueId, Shibboleth checks the actual scope against the authorized scope(s) in metadata. In the case of the SAML2 Persistent NameID and eduPersonTargetedID (which are equivalent), the software checks the NameQualifier XML attribute against the actual issuer. In any case, if the software detects a mismatch, the user identifier is not accepted and thus not added to the user’s session.

Other relying party software might do scope-checking, I don’t know. If your software (SAML or otherwise) does this, please log into Confluence and add a comment to the end of this article.

Exploring The World of Metadata

Global Interfederation has enabled a world of new opportunities and relationships in the trust fabric.  As a result of this expanded landscape, some new questions are likely to be asked.  How many Identity Providers are there in eduGAIN?  Which federation has the most Service Providers?  What is the most widely-published Service Provider across all research and education federations?  From which federation does this Service Provider originate?

These and many other questions can be answered using a number of metadata exploration tools that are now available to the InCommon community via its partnerships with REFEDS (Research and Education Federations) and eduGAIN.  In order to help members and operators of research and education federations, REFEDS built the Metadata Explorer Tool (MET).  This service is available at:

The MET application shows general statistics about the different types of SAML metadata entities published in each of the known R&E federations.  Available information includes:

  • Summary of federations

  • Summary of interfederations

  • Search for SAML entities

  • Details on each entity published, including what federations it’s published in human-readable versions of each entity’s SAML UI data elements, etc.

MET is a great tool that can help you understand global metadata, including IdPs and SPs that may not be published in eduGAIN.  That can help you to look for interesting services you may want to ask to have published in eduGAIN, and it might also help you understand how your own SAML entities (if you’re an InCommon Site Admin or Delegated Admin) are being republished.

In addition to MET, the GÉANT eduGAIN ops team has published a couple of tools that can help you find and understand SAML entities that are published in eduGAIN, at a higher level of detail.  The first of these is the eduGAIN entities database:  This tool lets you look for entities published in eduGAIN from any or all of the federations that are members of eduGAIN.  It gives an up-to-date view of the number of and types of entities published by eduGAIN, and lets you search on details such as federation of origin, entity category, support for SAML 2.0, etc.  One very interesting feature of this application is that it shows “entity clashes” —  SAML entity descriptors that originate in more than one federation, and are published to eduGAIN from more than one federation.  It will show which entity descriptor, of all available, was chosen for inclusion in the eduGAIN metadata.  That can help you understand potential problems such as metadata elements that may be missing from an entity descriptor imported from eduGAIN, that you think should be available, but are not.  While we’re talking about that, you should know that any entity that is published in InCommon will always take precedence — that is, if an entity is published in InCommon and eduGAIN, InCommon will always publish the InCommon version of the entity descriptor.

The other tool that eduGAIN provides is the eduGAIN Connectivity Check Service (ECCS):  This application lets you browse and search IdPs that are known to exist in global R&E federations, and highlights potential issues that these IdPs may have in offering service via eduGAIN.  For example, they may not be published in eduGAIN, or may have connectivity problems.

We hope you find these tools useful — if you have any feedback or questions, please send a note to

One final note:  If you find this information useful or interesting, or you are just generally interested in collaborating in the development of federations, we strongly encourage you to join the REFEDS community.  It’s open to all, and is somewhat analogous to an international version of the InCommon Participants list.  REFEDS is the primary place where the community defines international R&E federation practices, policies, standards, etc.  One good example is the global Research and Scholarship (R&S) entity category.  You can find out more about participating in REFEDS at:

Getting Ready for eduGAIN

As you may know, InCommon has joined eduGAIN and has begun making the policy and technical changes needed to participate in international interfederation. Those who manage identity providers and service providers in the InCommon Federation have choices to make as we approach February 15, 2016, which is when InCommon will begin importing and exporting metadata from and to eduGAIN at scale.

Please don’t wait until February to start thinking about how this major change affects you. Prior to February 15, there are a number of things to consider for each SAML deployment that consumes InCommon metadata:

  1. Once we start importing eduGAIN metadata, the size of the InCommon production aggregate will more than double. Is your deployment ready to consume such a large aggregate? To find out how to prepare for eduGAIN metadata, read the Getting Ready for eduGAIN wiki page.

  2. IdP metadata will be exported by default while SP metadata will be exported on demand. What are the implications of each choice? Are there guidelines as you consider whether to export your metadata? What does it mean if you decide not to export? Find out by reviewing the Getting Ready for eduGAIN wiki page.

If you are currently running Shibboleth IdP V2, now is a good time to begin upgrading to Shibboleth IdP V3. Although V3 is not strictly required for eduGAIN, V2 is quickly approaching end-of-life and InCommon recommends upgrading to V3 as soon as possible. For those who prefer structured help in upgrading to V3, or installing from scratch, InCommon has scheduled three Shibboleth workshops for the first half of 2016.

If you have any questions, please contact us at



meta-attribute is an abstract "above-the-wire" user attribute. Meta-attributes are used to unambiguously specify attribute requirements in deployment profiles, in attribute queries, and in SAML metadata. The meta-attribute concept is similar to the notion of "scope" in OpenID Connect.


Meta-attributes provide an unambiguous language for formulating attribute requirements in service (or client) metadata or in attribute queries. Other uses of meta-attributes include:

  • To standardize attributes (or claims) across multiple protocols (such as SAML Web Browser SSO and OpenID Connect)
  • To reconcile apparent conflicts among entity categories with competing attribute requirements
  • To provide flexibility to IdP operators when configuring attribute release policy rules
  • To make it easy to design usable consent interfaces

For discussion purposes, a Meta-Attribute Registry is defined in the appendix below. The registry defines groups of attributes—including both eduPerson attributes and OpenID Connect claims—as higher-level meta-attributes. Using meta-attributes, references to user attributes can be made in schema-independent fashion.

Requested Attributes in Metadata

Each meta-attribute in the Meta-Attribute Registry includes an example that shows how the meta-attribute is used in (SAML) metadata. A more comprehensive example follows, including sample Shibboleth IdP V3 configurations based on requested attributes in SP metadata.

Requesting Wire Attributes in Metadata

Suppose an SP has the following requested attributes in metadata:

<md:RequestedAttribute FriendlyName="eduPersonPrincipalName"

<md:RequestedAttribute FriendlyName="displayName"

<md:RequestedAttribute FriendlyName="mail"

Then a Shibboleth IdP with the following configuration will release the indicated wire attributes to the above SP:

<afp:AttributeFilterPolicy id="releaseEssentialAttributesToAnySP">

  <afp:PolicyRequirementRule xsi:type="basic:ANY"/>

  <!-- assuming ePPN is non-reassigned -->
  <afp:AttributeRule attributeID="eduPersonPrincipalName">
    <afp:PermitValueRule xsi:type="AttributeInMetadata"/>

  <afp:AttributeRule attributeID="displayName">
    <afp:PermitValueRule xsi:type="AttributeInMetadata"/>

  <afp:AttributeRule attributeID="email">
    <afp:PermitValueRule xsi:type="AttributeInMetadata"/>


An IdP so configured will not release attributes to an SP unless the indicated requested attributes are in SP metadata.

Unfortunately, requesting specific wire attributes in this way doesn't work very well in practice. Take eduPersonPrincipalName, for instance. The SP might be perfectly happy to receive eduPersonUniqueId in lieu of eduPersonPrincipalName but unfortunately there is no way to express this complex requirement in metadata. This is where meta-attributes come in handy.

Requesting Meta-Attributes in Metadata

Now suppose an SP has the following requested attributes in metadata:

<md:RequestedAttribute FriendlyName="metaPublicUserID"

<md:RequestedAttribute FriendlyName="metaPersonName"

<md:RequestedAttribute FriendlyName="metaEmailAddress"

Then two Shibboleth IdPs each with the following configurations will release the indicated wire attributes to the above SP:

<afp:AttributeFilterPolicy id="mapAndReleaseEssentialAttributesToAnySP">

  <afp:PolicyRequirementRule xsi:type="basic:ANY"/>

  <!-- assuming ePPN is non-reassigned -->
  <afp:AttributeRule attributeID="eduPersonPrincipalName">
    <afp:PermitValueRule xsi:type="AttributeInMetadata"

  <afp:AttributeRule attributeID="displayName">
    <afp:PermitValueRule xsi:type="AttributeInMetadata"

  <afp:AttributeRule attributeID="email">
    <afp:PermitValueRule xsi:type="AttributeInMetadata"

<afp:AttributeFilterPolicy id="mapAndReleaseEssentialAttributesToAnySP">

  <afp:PolicyRequirementRule xsi:type="basic:ANY"/>

  <!-- where ePUId is non-reassigned by definition -->
  <afp:AttributeRule attributeID="eduPersonUniqueId">
    <afp:PermitValueRule xsi:type="AttributeInMetadata"

  <afp:AttributeRule attributeID="givenName">
    <afp:PermitValueRule xsi:type="AttributeInMetadata"

  <afp:AttributeRule attributeID="surname">
    <afp:PermitValueRule xsi:type="AttributeInMetadata"

  <afp:AttributeRule attributeID="email">
    <afp:PermitValueRule xsi:type="AttributeInMetadata"


Note that both IdPs have an attribute release policy that relies on the same set of requested attributes, but the requested attributes are mapped to different wire attributes in each case.

Building a Better Entity Category

Let's see how meta-attributes can help us build an entity category that optimizes attribute release.

The primary purpose of a service category (i.e., a category of service providers) is to facilitate attribute release. Clearly IdPs won't release attributes to SPs just because there are requested attributes in metadata, but it may help. Consider the following hypothetical example of an entity category.

Let's begin by defining an attribute bundle consisting of just three meta-attributes:

  1. Public User Identifier
  2. Person Name
  3. Email Address

Note that the underlying user attributes in the bundle are a subset of what is commonly called directory information.

Suppose the following entity attribute signifies membership in a hypothetical entity category we'll call the Ready to Collaborate Category:

<!-- the ready-to-collaborate entity attribute for SPs -->
<saml:Attribute xmlns:saml="urn:oasis:names:tc:SAML:2.0:assertion"

An SP is a member of the Ready To Collaborate Category if it exhibits the ready-to-collaborate entity attribute in its metadata. An IdP that supports the Ready To Collaborate Category recognizes that entity attribute as follows:

<afp:AttributeFilterPolicy id="mapAndReleaseAttributesToAnyReadyToCollaborateSP">

  <afp:PolicyRequirementRule xsi:type="EntityAttributeExactMatch"

  <!-- assuming ePPN is non-reassigned -->
  <afp:AttributeRule attributeID="eduPersonPrincipalName">
    <afp:PermitValueRule xsi:type="AttributeInMetadata" onlyIfRequired="true"

  <afp:AttributeRule attributeID="displayName">
    <afp:PermitValueRule xsi:type="AttributeInMetadata" onlyIfRequired="true"

  <afp:AttributeRule attributeID="email">
    <afp:PermitValueRule xsi:type="AttributeInMetadata" onlyIfRequired="true"


The IdP releases attributes to an SP when two conditions are met:

  1. The SP is tagged with the ready-to-collaborate entity attribute (presumably put there by a federation operator)
  2. The SP has one or more of the designated meta-attributes in its metadata

As shown earlier, different IdPs can release different wire attributes as long as the IdP conforms to the requirements of the category (which depends on meta-attributes in the registry).

The above configuration is optimal in the following sense:

  • An attribute is released only if the corresponding <md:RequestedAttribute> element is decorated with an isRequired="true" XML attribute.
  • If an SP lists all three meta-attributes in metadata, an IdP that supports the category will release all three.
  • If an SP lists less than all three meta-attributes, then that's exactly what it gets.
  • If an SP lists no meta-attributes in metadata, it should expect to receive no attributes from a supporting IdP.
  • An SP may list more than the three meta-attributes in metadata, but a supporting IdP is not required to release them.

So the use of meta-attributes optimizes the attribute release process.



Meta-Attribute Registry

A simple registry of meta-attributes illustrates the concept.

User Identifier

FriendlyName: metaUserID

metaUserID is a persistent, non-reassigned identifier.

An Identity Provider (or Attribute Authority) is said to release a metaUserID when it releases one of the following attributes on the wire:

  1. eduPersonTargetedID

  2. eduPersonUniqueId

  3. eduPersonPrincipalName (if non-reassigned)

  4. OpenID Connect sub claim

A Service Provider is said to request a metaUserID when it does so directly, as shown in the following example.


Here is an example of an abstract metaUserID requested in Service Provider metadata:

<md:RequestedAttribute FriendlyName="metaUserID"

Public User Identifier

FriendlyName: metaPublicUserID

metaPublicUserID is a persistent, non-reassigned, non-targeted identifier.

An Identity Provider (or Attribute Authority) is said to release a metaPublicUserID when it releases one of the following attributes on the wire:

  1. eduPersonUniqueId

  2. eduPersonPrincipalName (if non-reassigned)

  3. OpenID Connect public sub claim

A Service Provider is said to request a metaPublicUserID when it does so directly, as shown in the following example.


Here is an example of an abstract metaPublicUserID requested in Service Provider metadata:

<md:RequestedAttribute FriendlyName="metaPublicUserID"

Person Name

FriendlyName: metaPersonName

metaPersonName is a human-readable name for the person (or subject) involved in a federated transaction.

An Identity Provider (or Attribute Authority) is said to release a metaPersonName when it releases at least one of the following attributes (or attribute combinations) on the wire:

  1. eduPerson displayName attribute

  2. OpenID Connect name claim

  3. Two eduPerson attributes: givenName + sn (surname)

  4. Two OpenID Connect claims: given_name + family_name

A Service Provider is said to request a metaPersonName when it does so directly, as shown in the following example.


Here is an example of an abstract metaPersonName requested in Service Provider metadata:

<md:RequestedAttribute FriendlyName="metaPersonName"

Email Address

FriendlyName: metaEmailAddress

metaEmailAddress is an electronic mail address for the person (or subject) involved in a federated transaction.

An Identity Provider (or Attribute Authority) is said to release a metaEmailAddress when it releases one of the following attributes on the wire:

  1. eduPerson mail attribute
  2. OpenID Connect email claim

A Service Provider is said to request a metaEmailAddress when it does so directly, as shown in the following example.


Here is an example of an abstract metaEmailAddress requested in Service Provider metadata:

<md:RequestedAttribute FriendlyName="metaEmailAddress"

R&S IdPs that support global R&S are shown in green on the Entity Categories info page.

Does your Identity Provider (IdP) support the Research & Scholarship Category of services in the InCommon Federation? If so, read on to find out how to migrate your IdP to global Research & Scholarship. If your IdP does not yet support the Research & Scholarship category, the following section shows you how.

Support Research & Scholarship

More info for prospective R&S IdPs: Research and Scholarship for IdPs

An IdP that supports the Research & Scholarship (R&S) category automatically releases the R&S Attribute Bundle to R&S SPs, that is, SPs tagged with the R&S entity attribute. These SPs have been vetted by InCommon to meet the requirements of R&S as outlined on the Research & Scholarship for SPs wiki page.

To support the R&S category, an IdP operator has at least two configuration options:

  1. Release the R&S attribute bundle to all R&S SPs, including R&S SPs in other federations

  2. Release the R&S attribute bundle to R&S SPs registered by InCommon only

In either case, all that’s required is a one-time change to your IdP’s attribute release policy. For more info about R&S, visit our wiki: Research and Scholarship for IdPs

R&S is for ALL users!

The R&S category benefits all organizations, even those organizations whose primary mission is not research. The R&S category includes generally useful collaborative services (such as wikis and file sharing services) that benefit all users, including students, faculty, and staff at colleges and universities of all sizes.

Migrate to Global Research & Scholarship

More info for existing R&S IdPs: Migrating an IdP to Global Research and Scholarship

Today more than 100 InCommon IdPs support the Research & Scholarship (R&S) category. If your IdP supports R&S, we invite you to migrate to global R&S, that is, release attributes to all R&S SPs, including R&S SPs in other federations.

To migrate an existing R&S IdP to global R&S, the IdP operator makes a small, one-time change to their IdP configuration to recognize the R&S entity attribute value. Since all R&S SPs (in all federations) meet the requirements of the REFEDS R&S Entity Category specification, an IdP configured to recognize the R&S tag releases attributes to all R&S SPs, including R&S SPs in other federations.

An IdP configuration SHOULD NOT rely on the R&S tag

If you are unable to support global R&S at this time, we strongly advise you to update your IdP configuration anyway since the use of the legacy R&S tag to control attribute release is deprecated. Eventually the legacy R&S tag will be removed from SP metadata. See the wiki topic linked below for more information.

For more info about global R&S, visit our wiki: Migrating an IdP to Global Research and Scholarship

The SAML V2.0 Metadata Extensions for Registration and Publication Information is a specification for a set of extension elements to SAML metadata. These elements are particularly important for the purposes of interfederation. For example, every entity descriptor exported to eduGAIN must include the globally unique identifier of the registrar that registered that entity descriptor. See the sidebar for a complete list of registrars currently exporting metadata to eduGAIN.

Registration Info

Since metadata registrars rely on a wide variety of operating principles, we expect some metadata consumers to care who the registrar is, at least in the short term. To accommodate this potential need, a globally unique identifier for the InCommon registrar will be inserted into metadata:

The RegistrationInfo Element
  <mdrpi:RegistrationInfo registrationAuthority=""/>

Global Registrar IDs































According to the MD-RPI specification, the above extension element (and therefore the registrar's ID) may be inserted either at the aggregate level or the entity level. To accommodate per-entity metadata, the <mdrpi:RegistrationInfo> element will be inserted at the entity level. Consequently, the introduction of the MD-RPI schema will necessarily touch every entity descriptor in metadata.

Besides the registrationAuthority XML attribute, the <mdrpi:RegistrationInfo> element has two other (optional) components:

  • the registrationInstant attribute

  • the <mdrpi:RegistrationPolicy> child element

This optional information will not be included in InCommon metadata, however. The rest of this section gives a rationale for this decision.

Since InCommon maintains a repository of every metadata aggregate ever published, the registrationInstant for each entity descriptor may be computed, but since the utility of this attribute has not been demonstrated, the computation (and subsequent publication) of the per-entity registrationInstant value will be omitted.

On the other hand, the <mdrpi:RegistrationPolicy> child element appears to be more interesting. Note, however, that the MD-RPI specification contains the following explicit requirement:

The URL MUST represent a single, immutable, policy document. Any changes made to an existing policy document MUST result in a new URL.

Consequently, the value of the <mdrpi:RegistrationPolicy> element is not particularly useful as a component of an IdP’s attribute release policy since the metadata registration practices of Federations worldwide are expected to change while registration procedures converge to a common standard. In particular, the InCommon Metadata Registration Practice Statement is subject to change, and therefore the <mdrpi:RegistrationPolicy> element represents a potential maintenance issue for IdP operators. For this reason, the element will not be included in entity metadata, at least not at this time.

Attribute Release Policy

As suggested earlier, the registrar ID may be used to formulate an IdP’s attribute release policy (although the extent to which IdP operators will actually want to do this is unknown). What follows are some practical considerations for deployers of the Shibboleth IdP software.

Unfortunately, Shibboleth IdP 2.x does not support the MD-RPI schema out-of-the-box. However, a plugin for Shibboleth IdP V2 that adds support for the registrationAuthority XML attribute has been developed by the UK Federation. An IdP outfitted with this plugin may be configured to release attributes based on the SP’s registrar. See the “Configuration Examples” section of the plugin documentation for specific examples.

In contrast, support for the MD-RPI schema is native to Shibboleth IdP 3.x. Documentation is still kind of sketchy but see the AttributeFilterConfiguration topic in the Shibboleth wiki for an example of the new syntax.

Publication Info

The MD-RPI specification also defines an <mdrpi:PublicationInfo> element with the following three XML attributes:

  1. publisher (required)

  2. creationInstant

  3. publicationId

The latter will be omitted from InCommon metadata but the other two are a welcome addition. In particular, the publisher XML attribute will take on the location of the metadata aggregate, which will positively identify the source of the metadata. This is useful for debugging and perhaps other purposes.

The PublicationInfo Element

Unlike the <mdrpi:RegistrationInfo> element, the <mdrpi:PublicationInfo> element is intended to be used exclusively on the root element of the metadata, which implies the latter element should appear at the aggregate level, not the entity level.

Migration Strategy

Metadata aggregates are brittle by their very nature (which is yet another reason to embrace per-entity metadata) especially when introducing new XML schema into the mix. So as to minimize the risk of breakage, we will first introduce the MD-RPI schema into the preview aggregate. If all goes well, we will later sync the preview aggregate with the well-known main aggregate (which is sometimes called the production aggregate even though all aggregates published by InCommon are production-quality aggregates). The final step is to sync the main aggregate with the fallback aggregate. See the Metadata Aggregates wiki page for more info about this process.

By the way, if you have a test deployment handy (either IdP or SP), by all means point it at the preview aggregate and leave it that way indefinitely



Federation Without Boundaries

What is interfederation? The goal of interfederation is to dissolve federation boundaries. If and when that goal is achieved, the term “interfederation” will simply disappear from use since “interfederation” will be indistinguishable from what we now call “federation.” In other words, interfederation is the new federation!

We’ve seen this happen before. Once upon a time, the terms “internet,” “intranet,” and “extranet” were widely used. As network boundaries blurred, that distinction mostly vanished, and with it the old terminology simply collapsed onto a single term: internet (with or without a capital “I” depending on your point of view). Exactly the same phenomenon will happen with the terms “federation” and “interfederation.” The latter is merely a short-term manifestation of a maturing federated model.

Interfederation is still very much in its infant stage. At this point in its evolution, the focus is on metadata aggregation. There are two well-known SAML metadata aggregator implementations used for interfederation purposes:

AFAIK the latter is used by eduGAIN to aggregate the metadata for R&E federations worldwide. So far 37 R&E federations have joined eduGAIN, and of those, 28 are actively exporting metadata.

InCommon joined eduGAIN in April 2014 amid great fanfare (see the press release) but the truth of the matter is that InCommon has been slow to interfederate. I’m happy to say that has been turned around, largely due to the efforts of Ian Young, perhaps the world’s foremost expert on metadata, metadata aggregation, and interfederation. With Ian’s help, InCommon is quietly making its way onto the interfederation scene.

For instance, InCommon is poised to announce a Per-Entity Metadata Pilot Study, an activity previously recommended by the Metadata Distribution WG. To support this pilot study, we’ve deployed an instance of a Metadata Query Server based on Ian’s mdq-server reference implementation of the Metadata Query Protocol.

In parallel with the pilot study, InCommon Operations will deploy new tooling based on the Shibboleth Metadata Aggregator. This tooling will allow us to ramp up our export activities, at which point all InCommon participants will be given the opportunity to export their entity metadata to eduGAIN.

To be eligible to participate in interfederation activities, an organization must be an InCommon participant in good standing. Beyond that, IdP and SP metadata exported to eduGAIN must meet certain basic requirements expected of all SAML deployments. Next week’s TechEx conference will give us an opportunity to discuss these requirements with the InCommon community. Please join us at TechEx in Indianapolis!

Asserting ePPN Across the Gateway

It is generally recognized that asserting scoped attributes across a gateway is problematic. Social gateways are particularly troublesome since few social IdPs assert an attribute that maps naturally to eduPersonPrincipalName, which is a scoped attribute known to be required by many RPs in the R&E space.

Executive Summary

  1. The user’s email address is a poor choice for eduPersonPrincipalName asserted by a gateway.
  2. The OpenID Connect subject identifier (sub) more accurately maps to eduPersonTargetedID or eduPersonUniqueId, not eduPersonPrincipalName.
  3. For a social gateway, the recommended value of eduPersonPrincipalName is
    where user@domain1 is the email address of the user, social_idp is the name of the social provider, and domain2 is a domain owned by the organization that owns and operates the gateway.


It is well known that eduPersonPrincipalName (ePPN) is a globally unique, persistent identifier for the user. For level-setting purposes, we begin with the following facts about persistent identifiers and scoped attributes.

Persistent Identifiers

  • Definition. A persistent identifier for the user is one that spans multiple SSO sessions.
  • Although ePPN is a persistent identifier, it is not intended to be permanent. Relying parties certainly prefer that ePPN remain stable but users can and do change their ePPN for a variety of reasons.
  • Although non-reassignment is a highly desirable property of any persistent identifier, ePPN deployments are not guaranteed to be non-reassigned (but often are since it is understood that a persistent, non-reassigned identifier is more valuable than one that is not).

Scoped Attributes

  • ePPN is the primary example of a scoped attribute.
  • ePPN is globally unique by virtue of its scope, which by convention is a DNS name.
  • The scope part of a scoped attribute indicates the asserting authority. This is why a scope is a DNS name by convention.
  • A trusted third party (such as a federation) ensures that the scopes listed in metadata are rooted in registered domains owned by the organization deploying the IdP.
  • Normally an IdP asserts a scoped attribute with a scope part for which the IdP is authoritative. Likewise an SP filters scoped attributes for which the IdP is not authoritative, at least by default.

Email Address as ePPN?

If you were an enterprise architect designing an identity management system from scratch, it would be in your best interest to define ePPN such that it was a routable email address. There are many reasons for this, not the least of which is the fact that SaaS services invariably use email address as a user ID.

That said, when mapping attributes across a social gateway, resist the urge to map the user’s email address to ePPN, even if the social IdP asserts email addresses known not to be re-assigned. Why? Because the right hand side of an email address asserted by a social IdP can be just about anything, which forces the RP to accept practically any scope from the corresponding gateway. That totally defeats the purpose of scoped attributes.

Consider Google, for example. Since a Google Apps subscriber provisions local email addresses in Google Apps (e.g.,, the Google IdP will assert arbitrary email addresses (not just email addresses). Thus mapping email address to ePPN is quite possibly the worst thing you could do.

The OIDC Sub Claim as ePPN?

The OpenID Connect (OIDC) subject identifier (sub) is an opaque, non-reassigned identifier for the user, scoped to the issuer. Coupled with the OIDC issuer identifier (iss) the sub claim is a stable, globally unique identifier for the user. As such the sub claim closely aligns with the SAML2 Persistent NameID or the equivalent eduPersonTargetedID attribute. Unlike eduPersonTargetedID, however, the sub claim is not a targeted identifier.

The sub claim is also closely aligned with eduPersonUniqueId. The latter, however, is a scoped attribute, which leads to complications. The obvious choice of scope value is but this scope MUST NOT be asserted in gateway metadata. An RP would have to carefully configure the handling of scope in its SAML software. To make matters worse, eduPersonUniqueId is new and not widely deployed, so one should expect little support for it in existing SAML implementations.

Finally, mapping the sub claim to ePPN is least desirable for the following reasons:

  • Both eduPersonTargetedID and eduPersonUniqueId are better suited to carry the sub claim.
  • Like eduPersonUniqueId, ePPN is a scoped attribute, with all the same problems.
  • The RP does not expect the left hand side of ePPN to be opaque.

All in all, the sub claim is perhaps best mapped to eduPersonTargetedID.

Best Practices for Gateway ePPNs

For a social gateway, the recommended value of ePPN is:


where user@domain1 is the email address of the user, social_idp is the name of the social provider, and domain2 is a domain owned by the organization that owns and operates the gateway. For example, my ePPN asserted by the Internet2 Google Gateway is:

Since Internet2 is a Google Apps for Education campus, I am also known as:

since Google will readily assert my Internet2 email address if I happen to log in via the Internet2 IdP.

What happens when an RP that has been using a central gateway chooses to run its own local gateway? In that case, a migration will be necessary since the scope on the ePPN will no doubt change. Thus the best choice of scope in the first place is a stable value that won’t change over time, regardless of who owns and operates the gateway.

For example, consider the Internet2 Google Gateway again. The scope was chosen because:

  1. Internet2 owns the registered domain, which is a required characteristic of all scopes in metadata.
  2. The subdomain makes it easy for Internet2 to support other social providers if and when the time comes. (For instance, a Facebook Gateway would have scope
  3. If the Internet2 Google Gateway were promoted to a centrally-run gateway for other (non-Internet2) services, the scope would not have to change.

Now observe that the Internet2 service does not use the Internet2 Google Gateway today…but it could. The best scope for this particular service would be since then the service could easily migrate to its own gateway in the future if desired. We could modify the existing gateway implementation to assert an ePPN with scope but we wouldn’t be able to assert that scope in metadata since Internet2 does not own the registered domain In that case, the SAML software protecting would have to be locally reconfigured to accept scoped attributes of the form from the Internet2 Google Gateway.

Research and Scholarship as a Catalyst for Interfederation

It’s been almost two and a half years since the InCommon Research & Scholarship (R&S) Category opened its doors. Today there are 23 R&S SPs, a small but important category of InCommon services. Perhaps more importantly, 90 InCommon IdPs have openly declared their support for R&S. Last month alone, 10 IdPs threw their hat into the ring, the most in any single month thus far. Could it be that R&S has finally reached critical mass?

These and other events make me optimistic that the answer to the previous question might be yes:

  1. The Research & Scholarship Entity Category was adopted by REFEDS [Feb 2014]
  2. InCommon officially joined the international eduGAIN consortium [Apr 2014]

Together these events place the Research & Scholarship Entity Category at the very center of the international stage.


In February 2014, exactly two years after the first InCommon Service Provider was accepted into the R&S program, the REFEDS steering committee approved and published its own REFEDS Research & Scholarship Entity Category specification. Modeled closely after the InCommon R&S Category definition, the REFEDS version enables R&S interoperability across more than 30 federations worldwide. At least a handful of these federations are implementing REFEDS R&S as we speak. More are expected to follow in the months ahead.

Here in the US, the migration to REFEDS R&S has already begun. To ensure compatibility with other federations, InCommon will ask R&S service owners to re-apply for R&S while focusing on the differences between InCommon R&S and REFEDS R&S (which fortunately are few). Once all the R&S SPs have migrated to REFEDS R&S, the R&S IdPs will be asked to make a small, one-time change to their attribute release policy. At that point the migration to the international REFEDS R&S standard will be complete.


The international eduGAIN service is the mechanism by which interfederation takes place. Federations worldwide export entity metadata to eduGAIN while those same federations turn around and import metadata from eduGAIN, adding the imported metadata to the aggregates distributed to their respective members.

InCommon officially joined eduGAIN in April 2014. We don’t yet export metadata to eduGAIN, but when we do, expect the R&S SPs to be among the first to go. Indeed, the InCommon R&S entities (both SPs and IdPs) are the best candidates for interfederation since they are already committed to usability, interoperability, security, and privacy.

The Critical Importance of R&S

Interfederation tends to magnify the strengths and weaknesses of the federated model, in the following sense. Suppose you’re an SP that wants to reach the widest possible audience, that is, the entire R&E community worldwide. Three things have to happen: 1) the SP has to consume all the IdP metadata in the world; 2) likewise all the IdPs in the world need to consume the SP’s metadata; and 3) all IdPs have to release the necessary attributes to the SP. Although (1) and (2) seem daunting, that is precisely the function of eduGAIN, and so there is hope.

No amount of metadata sharing will help realize (3), however. Attribute release is the number one problem in the InCommon Federation. It's why we invented R&S in the first place. At the interfederation level, attribute release is further complicated by international privacy laws. It is unlikely attributes will flow between countries without some agreements eventually being in place. This problem is being addressed on multiple fronts, including R&S. Although it’s barely more than an experiment at this point, the REFEDS Research & Scholarship Entity Category is the best chance we have of making interfederation a worthwhile exercise in the short term.

Phasing Out SHA-1

Phasing Out the Use of SHA-1

As of January 1, 2014, NIST disallows the use of the SHA-1 digest algorithm in conjunction with digital signatures (see: NIST SP 800-57 Part 1, Revision 3, July 2012, Tables 3 and 4). This was a major driver behind the Phase 1 Recommendations of the Metadata Distribution Working Group, which gave rise to the Phase 1 Implementation Plan, an effort that is nearing completion.

On December 18, 2013, InCommon began a migration process for phasing out the use of the SHA-1 digest algorithm within the Federation. The next milestone event in that migration process is scheduled to take place on June 30, 2014, as described in the next section.


SHA-1 and SAML Metadata

All SAML metadata distributed by InCommon is digitally signed for authenticity and integrity. The XML signature on the metadata uses a hashing algorithm, first to bulk-digest the XML metadata and then to sign the XML document itself. Although these are independent operations, a single hashing algorithm is often used for both purposes, and until recently, the SHA-1 hashing algorithm was used exclusively.

At this point in time (May 2014), all but one of the metadata aggregates published by InCommon are signed using the SHA-256 digest algorithm. The fallback aggregate, which still uses the SHA-1 digest algorithm, will be upgraded by the middle of this calendar year.

On June 30, 2014, the fallback metadata aggregate will be synced with the production metadata aggregate. That is, after June 30, all metadata aggregates published by the InCommon Federation will be signed using the SHA-256 digest algorithm. To avoid complications, all SAML deployments are strongly encouraged to migrate to the production aggregate or the preview aggregate now but no later than June 30, 2014.

A frequently asked question is: What will happen if I do nothing? One answer is: In 90% of the cases, if you do nothing, your deployment will continue to function as normal after June 30th. We say this because we are confident that the majority of SAML deployments in the InCommon Federation are compatible with SHA-256.

Instead of drilling down on the remaining 10%, consider this: We know with 100% certainty that the fallback metadata aggregate will be signed using a SHA-256 digest algorithm beginning on June 30th, so knowing nothing else but that simple fact, we conclude that all deployers are better off explicitly migrating to the production aggregate than they are doing nothing because all other things being equal (which they are) a controlled migration is always safer than a forced migration.

Indeed, migrating to the production aggregate is as simple as changing the URL in your metadata configuration. Go ahead, schedule that simple configuration change subject to whatever change management policy you have in place at your site. You’ll know within minutes if it’s going to work, and honestly, there is a very high probability it will Just Work. If it does, you’re home free because you can do the rest of the migration on your own time. If it doesn’t work, you can quickly back out the change and invoke Plan B.

All SAML deployments in the InCommon Federation should migrate to the production metadata aggregate (or the preview metadata aggregate) now. Recommended configuration options for Shibboleth and simpleSAMLphp are documented in the wiki.

So why does the simple migration outlined above work without bootstrapping an authentic copy of the new metadata signing certificate? Because a conforming metadata consumption process ignores all the certificate details except the public key bound to the certificate, and the public key has not changed.

For more information, see: Metadata Migration Process

Questions? Join this mailing list:

SHA-1 is not an encryption algorithm

Encryption implies the ability to decrypt (i.e., encryption by definition is reversible) but SHA-1 and SHA-256 are one-way hash functions. A hash function by definition is not reversible. So neither SHA-1 nor SHA-256 are encryption algorithms.

SHA-1 and X.509 Certificates

SAML entities (IdPs and SPs) manage at least two types of private keys: TLS keys and SAML keys. The corresponding public keys are bound to X.509 certificates, which are signed of course. Like the signature on XML metadata, the signature over an X.509 certificate relies on a digest algorithm, typically the SHA-1 digest algorithm.

SHA-1 and TLS Certificates

As you probably know, SAML endpoints in metadata are protected with TLS. In particular, browser-facing endpoints in metadata are associated with TLS certificates signed by a trusted CA (i.e., a CA trusted by your browser). Browser-facing TLS certificates do not appear in metadata and are therefore out of scope with respect to the Implementation Plan.

Endpoints in metadata that support the so-called back-channel protocols are associated with SAML certificates in metadata. These and other certificates in metadata are discussed in the next section.

Not surprisingly, the signature over a trusted (browser-facing) TLS certificate typically relies on the SHA-1 digest algorithm, and so the CA and browser vendors are also migrating to the SHA-2 family of digest algorithms. This will be the topic of a future blog post.

SHA-1 and SAML Certificates

The rest of this section concentrates on the public keys bound to certificates in SAML metadata.

A SAML entity is a secure web server that produces and/or consumes SAML messages. These SAML messages may be signed or encrypted using asymmetric crypto operations. A relying party uses a trusted public key in metadata to verify an XML signature or perform an XML encryption operation. These public keys are bound to long-lived, self-signed certificates in metadata. Like all X.509 certificates, the signature on a certificate in metadata uses a hashing algorithm, which is almost always the SHA-1 hashing algorithm.

Fortunately, there are absolutely no security considerations concerning the signature on certificates in metadata. In fact, the only concern is the actual public key bound to the certificate in metadata. Every other aspect of the certificate is ignored by the SAML software (or at least software that’s doing the Right Thing). That’s why InCommon recommends the use of long-lived, self-signed certificates in metadata: the focus is on the key, not the certificate.

Don't fiddle with the signature on self-signed certificates in metadata. Until the signature on a certificate in metadata actually becomes an interoperability issue, it is best to leave it alone.

SHA-1 and SAML Assertions

Like metadata, SAML messages may be digitally signed for authenticity and integrity. Any SAML entity (IdP or SP) may sign a message but in practice it is the IdP that wields a signing key since SAML assertions issued by the IdP MUST be signed. The SP uses the IdP’s public signing key in metadata to verify the signature on the assertion just-in-time, that is, when the assertion is presented to the SP by the browser user.

It is believed that most IdPs in the InCommon Federation are signing assertions using the SHA-1 digest algorithm. This is because most IdP deployments use the Shibboleth IdP software, which is known to support SHA-1 only, at least out of the box.

SAML deployers should take note of the following facts:

  1. If your SAML deployment can consume metadata signed using the SHA-256 digest algorithm, it very likely can consume SAML assertions signed using the SHA-256 digest algorithm.
  2. The ability to sign SAML assertions using the SHA-256 digest algorithm is not fully supported by the Shibboleth IdP software (but please read on).

Clearly the former is an enabler while the latter is an inhibitor. Without ample software support, InCommon can not realistically recommend a firm SHA-256 migration path for IdPs. The best we can do is to ensure ubiquitous SHA-256 support at the SP and then encourage individual IdPs to devise a migration plan that makes sense for them.

After June 30, 2014, almost all SPs in the InCommon Federation will support SHA-256. Given this, each IdP should decide, subject to its own unique set of circumstances, when to begin signing SAML assertions using the SHA-256 digest algorithm.

SimpleSAMLphp has good support for signing assertions using the SHA-256 digest algorithm. It can be configured to sign assertions on a per-SP basis, that is, a simpleSAMLphp IdP can sign assertions using the SHA-1 digest algorithm for some SPs and the SHA-256 digest algorithm for others. Therefore sites running the simpleSAMLphp IdP software can and should migrate to SHA-256 as soon as possible. A perfectly reasonable strategy would be to configure the use of SHA-256 by default but to fall back to SHA-1 for those few remaining SPs (perhaps external to InCommon) unable to handle SHA-256.

To enable SHA-2 support in the Shibboleth IdP, you can use a 3rd-party extension to enable SHA-2 capability for Shibboleth IdP versions 2.3 and later. Note that the Shibboleth configuration is all or none, either SHA-1 or SHA-256, so Shibboleth IdPs are advised to wait until after June 30, 2014, when all SPs will be consuming metadata signed using a SHA-256 digest algorithm.

Actually, it’s a bit more complicated than that for Shibboleth deployers. Shibboleth IdP v3 is due out later this calendar year and of course v3 will fully support the SHA-2 family of digest algorithms, including SHA-256. So the question for each Shibboleth deployer is whether to install the extension in their current deployment of Shibboleth IdP v2 or wait for Shibboleth IdP v3?

For those IdPs certified or looking to be certified in the Assurance Program, the Bronze and Silver Profiles require that "Cryptographic operations shall use Approved Algorithms" and since SHA-1 is no longer an "Approved Algorithm," it can not be used to meet the requirements of Bronze or Silver. However, the InCommon Assurance Program is working on releasing a variance to the requirements (called an Alternative Means) to permit the use of SHA-1 until January 15, 2015.

Be advised that if you intend to interoperate with a federal agency SP that requires Bronze or Silver, assertions MUST be signed using a SHA-2 digest algorithm. Such an IdP should begin the migration to SHA-256 immediately.