InCommon is pleased to announce general availability of version 3.0 of the InCommon Federation Manager. Release notes for this version of the Federation Manager may be found on the wiki.
This release introduces responsive design (targeted for use on mobile devices) for the Site Administrator functions and provides Site Administrators with a new "dashboard" from which they can easily determine the status of their metadata, as well as add and modify metadata. This simplifies the submission of metadata. More simplification is planned for upcoming releases. If you are interested in what features are targeted for future releases, please take a look at the Federation Manager Roadmap.
This roadmap and the Federation Manager releases this year were made possible by the InCommon dues increase enacted by InCommon steering in the fall of 2016. One target for that dues increase was 'hardening of InCommon infrastructure', and this work is part of that commitment.
The InCommon metadata aggregate is approaching 43MB and continues to grow. To compensate, InCommon Operations will enable HTTP Compression on the metadata server (md.incommon.org) as follows:
Friday, April 7: Preview Aggregate (InCommon-metadata-preview.xml)
Tuesday, April 11: IdP-only Aggregate (InCommon-metadata-idp-only.xml)
The remaining metadata aggregates (including the main production aggregate) will be enabled for HTTP Compression at a later date TBD.
There is nothing you need to do to realize the benefits of HTTP Compression. On or after the above date, if you are consuming either the Preview Aggregate or the IdP-only Aggregate, you may observe reduced bandwidth use and decreased download time during metadata refresh. This is especially true of the Preview Aggregate, which is just 8MB compressed.
The client and the server negotiate compression automatically. The server will send a compressed metadata aggregate if (and only if) your metadata client indicates support for compression in the HTTP request. Check your client software documentation for details.
If you have any concerns or experience any issues, please contact us at firstname.lastname@example.org
A Progress Report on the Migration to Shibboleth IdP V3
You’ve probably heard by now that Shibboleth IdP V2 reached end-of-life on July 31, 2016. Going forward, no bug fixes—not even security-related bug fixes—will be issued by the Shibboleth Project. In other words, Shibboleth IdP V2 is unsupported software.
Shibboleth IdP operators in the InCommon federation and other federations around the world are actively Upgrading to Shibboleth IdP V3. This article summarizes the progress of that effort over the last six (6) months.
All statistics in this article were compiled on January 2, 2017.
Here are some interesting facts:
Nearly 90% of the IdPs registered by InCommon are Shibboleth IdPs
Three-fourths of the world’s IdP deployments are based on the Shibboleth software
Four federations deploy three-fourths of the world’s Shibboleth IdPs
The same four federations account for three-fourths of the Shibboleth IdP V3 deployments worldwide
More than half (23/40) of the world’s federations each deploy fewer than three (3) Shibboleth IdPs
- Fifteen federations deploy no Shibboleth IdPs whatsoever
Read on for details!
Shibboleth IdPs in InCommon Metadata
Of the 451 IdPs registered by InCommon, 403 (89%) are Shibboleth IdPs. Of these, 226 (56%) are Shibboleth IdP V3 deployments.
In the last six (6) months, the number of V3 IdPs registered by InCommon has increased from 125 to 226. As a percentage, V3 deployments increased from 32% to 56% over the same time period.
|Number of IdPs||433||451|
|Number of Shibboleth IdPs||389 (90%)||403 (89%)|
|Number of Shibboleth IdP V3||125 (32%)||226 (56%)|
For more information: Lists of Shibboleth IdPs Registered by InCommon
Now let’s take a look at global migration activity by examining eduGAIN metadata.
Shibboleth IdPs in eduGAIN Metadata
Of the 2262 IdPs in eduGAIN metadata, 1690 (75%) are Shibboleth IdPs. Of these, 818 (48%) are Shibboleth IdP V3 deployments.
In the last six (6) months, the number of V3 IdPs in eduGAIN metadata has increased from 397 to 818. As a percentage, V3 deployments have nearly doubled over the same time period, increasing from 25% to 48%.
|Number of IdPs||2134||2262|
|Number of Shibboleth IdPs||1603 (75%)||1690 (75%)|
|Number of Shibboleth IdP V3||397 (25%)||818 (48%)|
For a complete analysis, including the number of Shibboleth IdP V3 deployments per registrar, see: Summary of Shibboleth IdPs in eduGAIN Metadata
For More Information
For the latest news and progress, visit this wiki page, which is updated regularly:
All of the data in this article were compiled using the Shibboleth IdP Probe, a script that probes a sequence of Shibboleth IdPs and determines the version of the Shibboleth software used by each deployment. The software can be downloaded from GitHub.
New and Emerging Methods of SAML Metadata Distribution
In September 2016, the Per-Entity Metadata Working Group, led by Scott Koranda, submitted an interim report to the InCommon Technical Advisory Committee with the following recommendation:
The working group finds and recommends that InCommon Operations proceed immediately with the design, creation, and delivery of a new InCommon metadata aggregate that contains only the metadata for identity providers (IdPs). The new IdP-only aggregate will help relieve issues some SPs face as the size of the existing InCommon metadata aggregates continues to grow.
Using the IdP-only metadata aggregate
Characteristics of the IdP-only Aggregate
The IdP-only metadata aggregate is approximately 16MB, which is about 42% of the size of the full InCommon aggregate.
As of October 10, the IdP-only metadata aggregate contains 2231 entity descriptors, of which 447 are registered by InCommon. Each entity descriptor contains an
<md:IDPSSODescriptor> child element. Seven (7) of the entities contain an
<md:SPSSODescriptor> child element as well.
For a complete up-to-date list of IdPs in InCommon metadata, see the List of IdP Display Names wiki page.
Benefits and Risks of the IdP-only Aggregate
What is per-entity metadata?
The SAML specification defines two entities: the Identity Provider (a producer of SAML assertions) and the Service Provider (a consumer of SAML assertions). A Service Provider requires the “metadata” of the Identity Provider (and vice versa). The metadata describe a SAML deployment, providing security, privacy, and interoperability to the relying party.
As a practical matter, SAML metadata is batch distributed as an aggregate of entity descriptors. With the proliferation of global aggregation services such as eduGAIN, the size of aggregates has grown dramatically, which is causing federations to re-examine existing methods of metadata distribution.
The term “per-entity metadata” refers to a single entity descriptor. The Metadata Query Protocol is an emerging standard that describes how to obtain per-entity metadata from a trusted oracle. Since the entity descriptor is the basic unit of policy and interoperability, this method of metadata distribution is both logical and efficient.
Since the IdP-only metadata aggregate is significantly smaller than the full aggregate, the former buys valuable time for service provider deployments—especially modestly provisioned deployments—until per-entity metadata becomes readily available. For one particular class of service providers, the IdP-only aggregate will continue to be essential infrastructure long after other InCommon deployments have migrated to per-entity metadata. The rest of this section explains why this is so.
The vast majority of SPs do not have a dynamic discovery interface (i.e., a discovery interface that depends on published metadata) and so these SPs will be able to leverage per-entity metadata without delay. In fact, many of these SPs depend on a small number of fixed IdPs so the migration to per-entity metadata will be straightforward for them.
On the other hand, for the relatively few SPs that implement a dynamic discovery interface, the benefit of per-entity metadata is less clear since these SPs currently require an aggregate for IdP discovery. We expect these SPs to consume the IdP-only aggregate until the community addresses the IdP discovery issue brought about by per-entity metadata.
Be aware that there is no fallback aggregate of IdP-only metadata. In that sense, there is some risk associated with the use of the IdP-only aggregate. If you must fall back, you will have no choice but to fall back to the full Fallback Aggregate described on the Metadata Aggregates wiki page.
The Future is Per-entity Metadata
The Per-Entity Metadata Working Group is expected to submit its final report to the InCommon TAC by November 2016, after the community has reviewed the report. We anticipate that the working group will recommend that InCommon Operations deploy a production-quality metadata query server and that all InCommon SAML deployments (except those SPs that implement a dynamic discovery interface as discussed above) migrate to per-entity metadata as soon as possible.
Eventually all SAML deployments will benefit from per-entity metadata. IdP deployers, in particular, are anxiously awaiting the arrival of a metadata query server, and we expect many IdPs will be among the first deployments to realize the benefits of per-entity metadata.
Two SAML implementations are known to support the Metadata Query Protocol: simpleSAMLphp and Shibboleth. (See the MDQ Client Software wiki page for more information.) In particular, support for the Metadata Query Protocol was introduced in version 3 of the Shibboleth IdP software. Shibboleth IdP deployments that have upgraded to Shibboleth IdP V3 will be among the first to migrate to per-entity metadata.
Shibboleth IdP V2 End-of-Life
Other SAML software will benefit from per-entity metadata as well. For example, Microsoft AD FS can be configured to retrieve a single entity descriptor from a metadata query server, which is a huge step in the right direction. The hope is that AD FS and other SAML implementations will eventually support the RESTful Metadata Query Protocol like simpleSAMLphp and Shibboleth.
Scoped User Identifiers
Recently a serious flaw was found in Office 365:
You should of course review the report and make your own determination but here’s a spoiler: The Office 365 application neglected to scope-check a user identifier, which allowed an arbitrary identity provider to assert any user identifier whatsoever and thereby gain unauthorized access to the application.
Here are a few lessons learned from the Office 365 vulnerability.
Lesson Learned #1
The Office 365 incident revolved around the following SAML attribute:
From a SAML perspective, there are a number of problems with the
IDPEmail attribute shown above but the point that matters most is: The
IDPEmail attribute is not an email address at all. Its value may look like an email address, but in fact, the
IDPEmail attribute corresponds to the User Principal Name (
userPrincipalName) of the existing account of the user in Azure AD.
That’s an important observation: The
IDPEmail attribute is an identifier, and moreover, it is actually used by the Office 365 application for access control. That leads directly to our next observation.
Lesson Learned #2
Some identifiers are explicitly scoped. In other cases, an identifier is implicitly scoped to the issuer. In all cases, a user identifier must be scope-checked by the relying party.
Some user identifiers are explicitly scoped. For example, the so-called scoped attributes
eduPersonUniqueId encode a “scope” directly in the attribute value:
In the above examples, the “scope” is “osu.edu” and “internet2.edu,” respectively. Only one identity provider (IdP) is authorized to assert the scope in each case, namely, the Ohio State University IdP and the Internet2 IdP, respectively. A relying party must check the asserted scope just-in-time and be prepared to reject an identifier if it does not pass the scope check.
So how does the relying party know what “scope” an IdP is authorized to assert? Trusted metadata, of course! If you search the InCommon metadata aggregate you will find the following
<shibmd:Scope> element is associated with the IdP authorized to assert that scope, namely, the Ohio State University IdP and the Internet2 IdP, respectively.
Other Scoped Identifiers
The SAML2 Persistent NameID is explicitly scoped but in a completely different manner. Here is an example:
The SAML2 Persistent NameID is asserted by the IdP as a child element of the
<saml2:Subject> element or the
eduPersonTargetedID attribute. In either case, the
NameQualifier XML attribute must be scope-checked, that is, it must match the actual issuer of the assertion. Otherwise the identifier should be discarded by default.
All of the examples thus far have been SAML attributes and identifiers but scope-checking is not a SAML issue per se. For instance, the OpenID Connect "sub" claim is a locally unique, non-reassigned identifier for the user (sub = subject). Here is an example of an ID token asserted by an OpenID provider:
By definition, the "sub" claim is implicitly scoped to the issuer. In practice, a relying party would store the pair (iss, sub) and subsequently scope-check an ID token by matching both the "iss" value and the "sub" value. This prevents an arbitrary OpenID provider from asserting an unauthorized "sub" value, intentionally or otherwise.
At this point we’ve looked at the following identifiers:
SAML2 Persistent NameID
OpenID Connect "sub" claim
These aren’t the only identifiers that need to be scope-checked, however. See the NameIdentifiers topic in the Shibboleth wiki for a more complete list.
All of the above identifiers have well-defined scope semantics. The
IDPEmail attribute, OTOH, is ill-suited for cross-domain access control. While its name and value suggest an email address, the underlying identifier (
userPrincipalName) has no documented scope semantics AFAIK.
Lesson Learned #3
Note well: The application developer must scope-check all identifiers asserted by untrusted 3rd parties. This is especially true if the identifier is used for access control. Failure to do so may lead to major security holes like the one reported in Office 365.
Of course this assumes the application relies on scoped identifiers to begin with. In particular, an application should never rely on an email address to identify a user. An email address is not scoped. For instance, the email address email@example.com may be legitimately asserted by any IdP. Conclusion: an email address makes a lousy user identifier
Explicitly scoped identifiers (such as
eduPersonUniqueId, and SAML2 Persistent NameID) are preferred since smart middleware can do the work for you. For instance, the Shibboleth SP software scope-checks the above identifiers by default. In the case of the scoped attributes
eduPersonUniqueId, Shibboleth checks the actual scope against the authorized scope(s) in metadata. In the case of the SAML2 Persistent NameID and
eduPersonTargetedID (which are equivalent), the software checks the
NameQualifier XML attribute against the actual issuer. In any case, if the software detects a mismatch, the user identifier is not accepted and thus not added to the user’s session.
Other relying party software might do scope-checking, I don’t know. If your software (SAML or otherwise) does this, please log into Confluence and add a comment to the end of this article.
Global Interfederation has enabled a world of new opportunities and relationships in the trust fabric. As a result of this expanded landscape, some new questions are likely to be asked. How many Identity Providers are there in eduGAIN? Which federation has the most Service Providers? What is the most widely-published Service Provider across all research and education federations? From which federation does this Service Provider originate?
These and many other questions can be answered using a number of metadata exploration tools that are now available to the InCommon community via its partnerships with REFEDS (Research and Education Federations) and eduGAIN. In order to help members and operators of research and education federations, REFEDS built the Metadata Explorer Tool (MET). This service is available at: https://met.refeds.org/
The MET application shows general statistics about the different types of SAML metadata entities published in each of the known R&E federations. Available information includes:
Summary of federations
Summary of interfederations
Search for SAML entities
Details on each entity published, including what federations it’s published in human-readable versions of each entity’s SAML UI data elements, etc.
MET is a great tool that can help you understand global metadata, including IdPs and SPs that may not be published in eduGAIN. That can help you to look for interesting services you may want to ask to have published in eduGAIN, and it might also help you understand how your own SAML entities (if you’re an InCommon Site Admin or Delegated Admin) are being republished.
In addition to MET, the GÉANT eduGAIN ops team has published a couple of tools that can help you find and understand SAML entities that are published in eduGAIN, at a higher level of detail. The first of these is the eduGAIN entities database: https://technical.edugain.org/entities. This tool lets you look for entities published in eduGAIN from any or all of the federations that are members of eduGAIN. It gives an up-to-date view of the number of and types of entities published by eduGAIN, and lets you search on details such as federation of origin, entity category, support for SAML 2.0, etc. One very interesting feature of this application is that it shows “entity clashes” — SAML entity descriptors that originate in more than one federation, and are published to eduGAIN from more than one federation. It will show which entity descriptor, of all available, was chosen for inclusion in the eduGAIN metadata. That can help you understand potential problems such as metadata elements that may be missing from an entity descriptor imported from eduGAIN, that you think should be available, but are not. While we’re talking about that, you should know that any entity that is published in InCommon will always take precedence — that is, if an entity is published in InCommon and eduGAIN, InCommon will always publish the InCommon version of the entity descriptor.
The other tool that eduGAIN provides is the eduGAIN Connectivity Check Service (ECCS): https://technical.edugain.org/eccs/. This application lets you browse and search IdPs that are known to exist in global R&E federations, and highlights potential issues that these IdPs may have in offering service via eduGAIN. For example, they may not be published in eduGAIN, or may have connectivity problems.
We hope you find these tools useful — if you have any feedback or questions, please send a note to firstname.lastname@example.org.
One final note: If you find this information useful or interesting, or you are just generally interested in collaborating in the development of federations, we strongly encourage you to join the REFEDS community. It’s open to all, and is somewhat analogous to an international version of the InCommon Participants list. REFEDS is the primary place where the community defines international R&E federation practices, policies, standards, etc. One good example is the global Research and Scholarship (R&S) entity category. You can find out more about participating in REFEDS at: https://refeds.org/
Getting Ready for eduGAIN
As you may know, InCommon has joined eduGAIN and has begun making the policy and technical changes needed to participate in international interfederation. Those who manage identity providers and service providers in the InCommon Federation have choices to make as we approach February 15, 2016, which is when InCommon will begin importing and exporting metadata from and to eduGAIN at scale.
Please don’t wait until February to start thinking about how this major change affects you. Prior to February 15, there are a number of things to consider for each SAML deployment that consumes InCommon metadata:
Once we start importing eduGAIN metadata, the size of the InCommon production aggregate will more than double. Is your deployment ready to consume such a large aggregate? To find out how to prepare for eduGAIN metadata, read the Getting Ready for eduGAIN wiki page.
IdP metadata will be exported by default while SP metadata will be exported on demand. What are the implications of each choice? Are there guidelines as you consider whether to export your metadata? What does it mean if you decide not to export? Find out by reviewing the Getting Ready for eduGAIN wiki page.
If you are currently running Shibboleth IdP V2, now is a good time to begin upgrading to Shibboleth IdP V3. Although V3 is not strictly required for eduGAIN, V2 is quickly approaching end-of-life and InCommon recommends upgrading to V3 as soon as possible. For those who prefer structured help in upgrading to V3, or installing from scratch, InCommon has scheduled three Shibboleth workshops for the first half of 2016.
A meta-attribute is an abstract "above-the-wire" user attribute. Meta-attributes are used to unambiguously specify attribute requirements in deployment profiles, in attribute queries, and in SAML metadata. The meta-attribute concept is similar to the notion of "scope" in OpenID Connect.
Meta-attributes provide an unambiguous language for formulating attribute requirements in service (or client) metadata or in attribute queries. Other uses of meta-attributes include:
- To standardize attributes (or claims) across multiple protocols (such as SAML Web Browser SSO and OpenID Connect)
- To reconcile apparent conflicts among entity categories with competing attribute requirements
- To provide flexibility to IdP operators when configuring attribute release policy rules
- To make it easy to design usable consent interfaces
For discussion purposes, a Meta-Attribute Registry is defined in the appendix below. The registry defines groups of attributes—including both eduPerson attributes and OpenID Connect claims—as higher-level meta-attributes. Using meta-attributes, references to user attributes can be made in schema-independent fashion.
Requested Attributes in Metadata
Each meta-attribute in the Meta-Attribute Registry includes an example that shows how the meta-attribute is used in (SAML) metadata. A more comprehensive example follows, including sample Shibboleth IdP V3 configurations based on requested attributes in SP metadata.
Requesting Wire Attributes in Metadata
Suppose an SP has the following requested attributes in metadata:
Then a Shibboleth IdP with the following configuration will release the indicated wire attributes to the above SP:
An IdP so configured will not release attributes to an SP unless the indicated requested attributes are in SP metadata.
Unfortunately, requesting specific wire attributes in this way doesn't work very well in practice. Take
eduPersonPrincipalName, for instance. The SP might be perfectly happy to receive
eduPersonUniqueId in lieu of
eduPersonPrincipalName but unfortunately there is no way to express this complex requirement in metadata. This is where meta-attributes come in handy.
Requesting Meta-Attributes in Metadata
Now suppose an SP has the following requested attributes in metadata:
Then two Shibboleth IdPs each with the following configurations will release the indicated wire attributes to the above SP:
Note that both IdPs have an attribute release policy that relies on the same set of requested attributes, but the requested attributes are mapped to different wire attributes in each case.
Building a Better Entity Category
Let's see how meta-attributes can help us build an entity category that optimizes attribute release.
The primary purpose of a service category (i.e., a category of service providers) is to facilitate attribute release. Clearly IdPs won't release attributes to SPs just because there are requested attributes in metadata, but it may help. Consider the following hypothetical example of an entity category.
Let's begin by defining an attribute bundle consisting of just three meta-attributes:
- Public User Identifier
- Person Name
- Email Address
Note that the underlying user attributes in the bundle are a subset of what is commonly called directory information.
Suppose the following entity attribute signifies membership in a hypothetical entity category we'll call the Ready to Collaborate Category:
An SP is a member of the Ready To Collaborate Category if it exhibits the
ready-to-collaborate entity attribute in its metadata. An IdP that supports the Ready To Collaborate Category recognizes that entity attribute as follows:
The IdP releases attributes to an SP when two conditions are met:
- The SP is tagged with the
ready-to-collaborateentity attribute (presumably put there by a federation operator)
- The SP has one or more of the designated meta-attributes in its metadata
As shown earlier, different IdPs can release different wire attributes as long as the IdP conforms to the requirements of the category (which depends on meta-attributes in the registry).
The above configuration is optimal in the following sense:
- An attribute is released only if the corresponding
<md:RequestedAttribute>element is decorated with an
- If an SP lists all three meta-attributes in metadata, an IdP that supports the category will release all three.
- If an SP lists less than all three meta-attributes, then that's exactly what it gets.
- If an SP lists no meta-attributes in metadata, it should expect to receive no attributes from a supporting IdP.
- An SP may list more than the three meta-attributes in metadata, but a supporting IdP is not required to release them.
So the use of meta-attributes optimizes the attribute release process.
A simple registry of meta-attributes illustrates the concept.
metaUserID is a persistent, non-reassigned identifier.
An Identity Provider (or Attribute Authority) is said to release a
metaUserID when it releases one of the following attributes on the wire:
- OpenID Connect
A Service Provider is said to request a
metaUserID when it does so directly, as shown in the following example.
Here is an example of an abstract
metaUserID requested in Service Provider metadata:
Public User Identifier
metaPublicUserID is a persistent, non-reassigned, non-targeted identifier.
An Identity Provider (or Attribute Authority) is said to release a
metaPublicUserID when it releases one of the following attributes on the wire:
- OpenID Connect public
A Service Provider is said to request a
metaPublicUserID when it does so directly, as shown in the following example.
Here is an example of an abstract
metaPublicUserID requested in Service Provider metadata:
metaPersonName is a human-readable name for the person (or subject) involved in a federated transaction.
An Identity Provider (or Attribute Authority) is said to release a
metaPersonName when it releases at least one of the following attributes (or attribute combinations) on the wire:
Two eduPerson attributes:
- Two OpenID Connect claims:
A Service Provider is said to request a
metaPersonName when it does so directly, as shown in the following example.
Here is an example of an abstract
metaPersonName requested in Service Provider metadata:
metaEmailAddress is an electronic mail address for the person (or subject) involved in a federated transaction.
An Identity Provider (or Attribute Authority) is said to release a
metaEmailAddress when it releases one of the following attributes on the wire:
- OpenID Connect
A Service Provider is said to request a
metaEmailAddress when it does so directly, as shown in the following example.
Here is an example of an abstract
metaEmailAddress requested in Service Provider metadata:
Does your Identity Provider (IdP) support the Research & Scholarship Category of services in the InCommon Federation? If so, read on to find out how to migrate your IdP to global Research & Scholarship. If your IdP does not yet support the Research & Scholarship category, the following section shows you how.
Support Research & Scholarship
An IdP that supports the Research & Scholarship (R&S) category automatically releases the R&S Attribute Bundle to R&S SPs, that is, SPs tagged with the R&S entity attribute. These SPs have been vetted by InCommon to meet the requirements of R&S as outlined on the Research & Scholarship for SPs wiki page.
To support the R&S category, an IdP operator has at least two configuration options:
Release the R&S attribute bundle to all R&S SPs, including R&S SPs in other federations
Release the R&S attribute bundle to R&S SPs registered by InCommon only
In either case, all that’s required is a one-time change to your IdP’s attribute release policy. For more info about R&S, visit our wiki: Research and Scholarship for IdPs
R&S is for ALL users!
Migrate to Global Research & Scholarship
Today more than 100 InCommon IdPs support the Research & Scholarship (R&S) category. If your IdP supports R&S, we invite you to migrate to global R&S, that is, release attributes to all R&S SPs, including R&S SPs in other federations.
To migrate an existing R&S IdP to global R&S, the IdP operator makes a small, one-time change to their IdP configuration to recognize the refeds.org R&S entity attribute value. Since all R&S SPs (in all federations) meet the requirements of the REFEDS R&S Entity Category specification, an IdP configured to recognize the refeds.org R&S tag releases attributes to all R&S SPs, including R&S SPs in other federations.
An IdP configuration SHOULD NOT rely on the incommon.org R&S tag
If you are unable to support global R&S at this time, we strongly advise you to update your IdP configuration anyway since the use of the legacy incommon.org R&S tag to control attribute release is deprecated. Eventually the legacy incommon.org R&S tag will be removed from SP metadata. See the wiki topic linked below for more information.
For more info about global R&S, visit our wiki: Migrating an IdP to Global Research and Scholarship
The SAML V2.0 Metadata Extensions for Registration and Publication Information is a specification for a set of extension elements to SAML metadata. These elements are particularly important for the purposes of interfederation. For example, every entity descriptor exported to eduGAIN must include the globally unique identifier of the registrar that registered that entity descriptor. See the sidebar for a complete list of registrars currently exporting metadata to eduGAIN.
Since metadata registrars rely on a wide variety of operating principles, we expect some metadata consumers to care who the registrar is, at least in the short term. To accommodate this potential need, a globally unique identifier for the InCommon registrar will be inserted into metadata:
According to the MD-RPI specification, the above extension element (and therefore the registrar's ID) may be inserted either at the aggregate level or the entity level. To accommodate per-entity metadata, the
<mdrpi:RegistrationInfo> element will be inserted at the entity level. Consequently, the introduction of the MD-RPI schema will necessarily touch every entity descriptor in metadata.
registrationAuthority XML attribute, the
<mdrpi:RegistrationInfo> element has two other (optional) components:
This optional information will not be included in InCommon metadata, however. The rest of this section gives a rationale for this decision.
Since InCommon maintains a repository of every metadata aggregate ever published, the
registrationInstant for each entity descriptor may be computed, but since the utility of this attribute has not been demonstrated, the computation (and subsequent publication) of the per-entity
registrationInstant value will be omitted.
On the other hand, the
<mdrpi:RegistrationPolicy> child element appears to be more interesting. Note, however, that the MD-RPI specification contains the following explicit requirement:
The URL MUST represent a single, immutable, policy document. Any changes made to an existing policy document MUST result in a new URL.
Consequently, the value of the
<mdrpi:RegistrationPolicy> element is not particularly useful as a component of an IdP’s attribute release policy since the metadata registration practices of Federations worldwide are expected to change while registration procedures converge to a common standard. In particular, the InCommon Metadata Registration Practice Statement is subject to change, and therefore the
<mdrpi:RegistrationPolicy> element represents a potential maintenance issue for IdP operators. For this reason, the element will not be included in entity metadata, at least not at this time.
Attribute Release Policy
As suggested earlier, the registrar ID may be used to formulate an IdP’s attribute release policy (although the extent to which IdP operators will actually want to do this is unknown). What follows are some practical considerations for deployers of the Shibboleth IdP software.
Unfortunately, Shibboleth IdP 2.x does not support the MD-RPI schema out-of-the-box. However, a plugin for Shibboleth IdP V2 that adds support for the
registrationAuthority XML attribute has been developed by the UK Federation. An IdP outfitted with this plugin may be configured to release attributes based on the SP’s registrar. See the “Configuration Examples” section of the plugin documentation for specific examples.
In contrast, support for the MD-RPI schema is native to Shibboleth IdP 3.x. Documentation is still kind of sketchy but see the AttributeFilterConfiguration topic in the Shibboleth wiki for an example of the new syntax.
The MD-RPI specification also defines an
<mdrpi:PublicationInfo> element with the following three XML attributes:
The latter will be omitted from InCommon metadata but the other two are a welcome addition. In particular, the publisher XML attribute will take on the location of the metadata aggregate, which will positively identify the source of the metadata. This is useful for debugging and perhaps other purposes.
<mdrpi:RegistrationInfo> element, the
<mdrpi:PublicationInfo> element is intended to be used exclusively on the root element of the metadata, which implies the latter element should appear at the aggregate level, not the entity level.
Metadata aggregates are brittle by their very nature (which is yet another reason to embrace per-entity metadata) especially when introducing new XML schema into the mix. So as to minimize the risk of breakage, we will first introduce the MD-RPI schema into the preview aggregate. If all goes well, we will later sync the preview aggregate with the well-known main aggregate (which is sometimes called the production aggregate even though all aggregates published by InCommon are production-quality aggregates). The final step is to sync the main aggregate with the fallback aggregate. See the Metadata Aggregates wiki page for more info about this process.
By the way, if you have a test deployment handy (either IdP or SP), by all means point it at the preview aggregate and leave it that way indefinitely
Federation Without Boundaries
What is interfederation? The goal of interfederation is to dissolve federation boundaries. If and when that goal is achieved, the term “interfederation” will simply disappear from use since “interfederation” will be indistinguishable from what we now call “federation.” In other words, interfederation is the new federation!
We’ve seen this happen before. Once upon a time, the terms “internet,” “intranet,” and “extranet” were widely used. As network boundaries blurred, that distinction mostly vanished, and with it the old terminology simply collapsed onto a single term: internet (with or without a capital “I” depending on your point of view). Exactly the same phenomenon will happen with the terms “federation” and “interfederation.” The latter is merely a short-term manifestation of a maturing federated model.
Interfederation is still very much in its infant stage. At this point in its evolution, the focus is on metadata aggregation. There are two well-known SAML metadata aggregator implementations used for interfederation purposes:
AFAIK the latter is used by eduGAIN to aggregate the metadata for R&E federations worldwide. So far 37 R&E federations have joined eduGAIN, and of those, 28 are actively exporting metadata.
InCommon joined eduGAIN in April 2014 amid great fanfare (see the press release) but the truth of the matter is that InCommon has been slow to interfederate. I’m happy to say that has been turned around, largely due to the efforts of Ian Young, perhaps the world’s foremost expert on metadata, metadata aggregation, and interfederation. With Ian’s help, InCommon is quietly making its way onto the interfederation scene.
For instance, InCommon is poised to announce a Per-Entity Metadata Pilot Study, an activity previously recommended by the Metadata Distribution WG. To support this pilot study, we’ve deployed an instance of a Metadata Query Server based on Ian’s mdq-server reference implementation of the Metadata Query Protocol.
In parallel with the pilot study, InCommon Operations will deploy new tooling based on the Shibboleth Metadata Aggregator. This tooling will allow us to ramp up our export activities, at which point all InCommon participants will be given the opportunity to export their entity metadata to eduGAIN.
To be eligible to participate in interfederation activities, an organization must be an InCommon participant in good standing. Beyond that, IdP and SP metadata exported to eduGAIN must meet certain basic requirements expected of all SAML deployments. Next week’s TechEx conference will give us an opportunity to discuss these requirements with the InCommon community. Please join us at TechEx in Indianapolis!
Asserting ePPN Across the Gateway
It is generally recognized that asserting scoped attributes across a gateway is problematic. Social gateways are particularly troublesome since few social IdPs assert an attribute that maps naturally to
eduPersonPrincipalName, which is a scoped attribute known to be required by many RPs in the R&E space.
- The user’s email address is a poor choice for
eduPersonPrincipalNameasserted by a gateway.
- The OpenID Connect subject identifier (sub) more accurately maps to
- For a social gateway, the recommended value of
user@domain1is the email address of the user,
social_idpis the name of the social provider, and
domain2is a domain owned by the organization that owns and operates the gateway.
It is well known that
ePPN) is a globally unique, persistent identifier for the user. For level-setting purposes, we begin with the following facts about persistent identifiers and scoped attributes.
- Definition. A persistent identifier for the user is one that spans multiple SSO sessions.
ePPNis a persistent identifier, it is not intended to be permanent. Relying parties certainly prefer that
ePPNremain stable but users can and do change their
ePPNfor a variety of reasons.
- Although non-reassignment is a highly desirable property of any persistent identifier,
ePPNdeployments are not guaranteed to be non-reassigned (but often are since it is understood that a persistent, non-reassigned identifier is more valuable than one that is not).
ePPNis the primary example of a scoped attribute.
ePPNis globally unique by virtue of its scope, which by convention is a DNS name.
- The scope part of a scoped attribute indicates the asserting authority. This is why a scope is a DNS name by convention.
- A trusted third party (such as a federation) ensures that the scopes listed in metadata are rooted in registered domains owned by the organization deploying the IdP.
- Normally an IdP asserts a scoped attribute with a scope part for which the IdP is authoritative. Likewise an SP filters scoped attributes for which the IdP is not authoritative, at least by default.
Email Address as ePPN?
If you were an enterprise architect designing an identity management system from scratch, it would be in your best interest to define
ePPN such that it was a routable email address. There are many reasons for this, not the least of which is the fact that SaaS services invariably use email address as a user ID.
That said, when mapping attributes across a social gateway, resist the urge to map the user’s email address to
ePPN, even if the social IdP asserts email addresses known not to be re-assigned. Why? Because the right hand side of an email address asserted by a social IdP can be just about anything, which forces the RP to accept practically any scope from the corresponding gateway. That totally defeats the purpose of scoped attributes.
Consider Google, for example. Since a Google Apps subscriber provisions local email addresses in Google Apps (e.g.,
email@example.com), the Google IdP will assert arbitrary email addresses (not just
@gmail.com email addresses). Thus mapping email address to
ePPN is quite possibly the worst thing you could do.
The OIDC Sub Claim as ePPN?
The OpenID Connect (OIDC) subject identifier (
sub) is an opaque, non-reassigned identifier for the user, scoped to the issuer. Coupled with the OIDC issuer identifier (
sub claim is a stable, globally unique identifier for the user. As such the
sub claim closely aligns with the SAML2 Persistent NameID or the equivalent
eduPersonTargetedID attribute. Unlike
eduPersonTargetedID, however, the
sub claim is not a targeted identifier.
sub claim is also closely aligned with
eduPersonUniqueId. The latter, however, is a scoped attribute, which leads to complications. The obvious choice of scope value is
@google.com but this scope MUST NOT be asserted in gateway metadata. An RP would have to carefully configure the handling of scope
@google.com in its SAML software. To make matters worse,
eduPersonUniqueId is new and not widely deployed, so one should expect little support for it in existing SAML implementations.
Finally, mapping the
sub claim to
ePPN is least desirable for the following reasons:
eduPersonUniqueIdare better suited to carry the
ePPNis a scoped attribute, with all the same problems.
- The RP does not expect the left hand side of
ePPNto be opaque.
All in all, the
sub claim is perhaps best mapped to
Best Practices for Gateway ePPNs
For a social gateway, the recommended value of
user@domain1 is the email address of the user,
social_idp is the name of the social provider, and
domain2 is a domain owned by the organization that owns and operates the gateway. For example, my
ePPN asserted by the Internet2 Google Gateway is:
Since Internet2 is a Google Apps for Education campus, I am also known as:
since Google will readily assert my Internet2 email address
firstname.lastname@example.org if I happen to log in via the Internet2 IdP.
What happens when an RP that has been using a central gateway chooses to run its own local gateway? In that case, a migration will be necessary since the scope on the
ePPN will no doubt change. Thus the best choice of scope in the first place is a stable value that won’t change over time, regardless of who owns and operates the gateway.
For example, consider the Internet2 Google Gateway again. The scope
@google.incommon.org was chosen because:
- Internet2 owns the registered domain
incommon.org, which is a required characteristic of all scopes in metadata.
- The subdomain
google.incommon.orgmakes it easy for Internet2 to support other social providers if and when the time comes. (For instance, a Facebook Gateway would have scope
- If the Internet2 Google Gateway were promoted to a centrally-run gateway for other (non-Internet2) services, the scope would not have to change.
Now observe that the Internet2 service
wiki.shibboleth.net does not use the Internet2 Google Gateway today…but it could. The best scope for this particular service would be
@google.shibboleth.net since then the service could easily migrate to its own gateway in the future if desired. We could modify the existing gateway implementation to assert an
ePPN with scope
@google.shibboleth.net but we wouldn’t be able to assert that scope in metadata since Internet2 does not own the registered domain
shibboleth.net. In that case, the SAML software protecting
wiki.shibboleth.net would have to be locally reconfigured to accept scoped attributes of the form
email@example.com from the Internet2 Google Gateway.
Research and Scholarship as a Catalyst for Interfederation
It’s been almost two and a half years since the InCommon Research & Scholarship (R&S) Category opened its doors. Today there are 23 R&S SPs, a small but important category of InCommon services. Perhaps more importantly, 90 InCommon IdPs have openly declared their support for R&S. Last month alone, 10 IdPs threw their hat into the ring, the most in any single month thus far. Could it be that R&S has finally reached critical mass?
These and other events make me optimistic that the answer to the previous question might be yes:
- The Research & Scholarship Entity Category was adopted by REFEDS [Feb 2014]
- InCommon officially joined the international eduGAIN consortium [Apr 2014]
Together these events place the Research & Scholarship Entity Category at the very center of the international stage.
In February 2014, exactly two years after the first InCommon Service Provider was accepted into the R&S program, the REFEDS steering committee approved and published its own REFEDS Research & Scholarship Entity Category specification. Modeled closely after the InCommon R&S Category definition, the REFEDS version enables R&S interoperability across more than 30 federations worldwide. At least a handful of these federations are implementing REFEDS R&S as we speak. More are expected to follow in the months ahead.
Here in the US, the migration to REFEDS R&S has already begun. To ensure compatibility with other federations, InCommon will ask R&S service owners to re-apply for R&S while focusing on the differences between InCommon R&S and REFEDS R&S (which fortunately are few). Once all the R&S SPs have migrated to REFEDS R&S, the R&S IdPs will be asked to make a small, one-time change to their attribute release policy. At that point the migration to the international REFEDS R&S standard will be complete.
The international eduGAIN service is the mechanism by which interfederation takes place. Federations worldwide export entity metadata to eduGAIN while those same federations turn around and import metadata from eduGAIN, adding the imported metadata to the aggregates distributed to their respective members.
InCommon officially joined eduGAIN in April 2014. We don’t yet export metadata to eduGAIN, but when we do, expect the R&S SPs to be among the first to go. Indeed, the InCommon R&S entities (both SPs and IdPs) are the best candidates for interfederation since they are already committed to usability, interoperability, security, and privacy.
The Critical Importance of R&S
Interfederation tends to magnify the strengths and weaknesses of the federated model, in the following sense. Suppose you’re an SP that wants to reach the widest possible audience, that is, the entire R&E community worldwide. Three things have to happen: 1) the SP has to consume all the IdP metadata in the world; 2) likewise all the IdPs in the world need to consume the SP’s metadata; and 3) all IdPs have to release the necessary attributes to the SP. Although (1) and (2) seem daunting, that is precisely the function of eduGAIN, and so there is hope.
No amount of metadata sharing will help realize (3), however. Attribute release is the number one problem in the InCommon Federation. It's why we invented R&S in the first place. At the interfederation level, attribute release is further complicated by international privacy laws. It is unlikely attributes will flow between countries without some agreements eventually being in place. This problem is being addressed on multiple fronts, including R&S. Although it’s barely more than an experiment at this point, the REFEDS Research & Scholarship Entity Category is the best chance we have of making interfederation a worthwhile exercise in the short term.
Phasing Out the Use of SHA-1
As of January 1, 2014, NIST disallows the use of the SHA-1 digest algorithm in conjunction with digital signatures (see: NIST SP 800-57 Part 1, Revision 3, July 2012, Tables 3 and 4). This was a major driver behind the Phase 1 Recommendations of the Metadata Distribution Working Group, which gave rise to the Phase 1 Implementation Plan, an effort that is nearing completion.
On December 18, 2013, InCommon began a migration process for phasing out the use of the SHA-1 digest algorithm within the Federation. The next milestone event in that migration process is scheduled to take place on June 30, 2014, as described in the next section.
SHA-1 and SAML Metadata
All SAML metadata distributed by InCommon is digitally signed for authenticity and integrity. The XML signature on the metadata uses a hashing algorithm, first to bulk-digest the XML metadata and then to sign the XML document itself. Although these are independent operations, a single hashing algorithm is often used for both purposes, and until recently, the SHA-1 hashing algorithm was used exclusively.
At this point in time (May 2014), all but one of the metadata aggregates published by InCommon are signed using the SHA-256 digest algorithm. The fallback aggregate, which still uses the SHA-1 digest algorithm, will be upgraded by the middle of this calendar year.
On June 30, 2014, the fallback metadata aggregate will be synced with the production metadata aggregate. That is, after June 30, all metadata aggregates published by the InCommon Federation will be signed using the SHA-256 digest algorithm. To avoid complications, all SAML deployments are strongly encouraged to migrate to the production aggregate or the preview aggregate now but no later than June 30, 2014.
A frequently asked question is: What will happen if I do nothing? One answer is: In 90% of the cases, if you do nothing, your deployment will continue to function as normal after June 30th. We say this because we are confident that the majority of SAML deployments in the InCommon Federation are compatible with SHA-256.
Instead of drilling down on the remaining 10%, consider this: We know with 100% certainty that the fallback metadata aggregate will be signed using a SHA-256 digest algorithm beginning on June 30th, so knowing nothing else but that simple fact, we conclude that all deployers are better off explicitly migrating to the production aggregate than they are doing nothing because all other things being equal (which they are) a controlled migration is always safer than a forced migration.
Indeed, migrating to the production aggregate is as simple as changing the URL in your metadata configuration. Go ahead, schedule that simple configuration change subject to whatever change management policy you have in place at your site. You’ll know within minutes if it’s going to work, and honestly, there is a very high probability it will Just Work. If it does, you’re home free because you can do the rest of the migration on your own time. If it doesn’t work, you can quickly back out the change and invoke Plan B.
So why does the simple migration outlined above work without bootstrapping an authentic copy of the new metadata signing certificate? Because a conforming metadata consumption process ignores all the certificate details except the public key bound to the certificate, and the public key has not changed.
For more information, see: Metadata Migration Process
Questions? Join this mailing list: https://lists.incommon.org/sympa/info/metadata-support
SHA-1 and X.509 Certificates
SAML entities (IdPs and SPs) manage at least two types of private keys: TLS keys and SAML keys. The corresponding public keys are bound to X.509 certificates, which are signed of course. Like the signature on XML metadata, the signature over an X.509 certificate relies on a digest algorithm, typically the SHA-1 digest algorithm.
SHA-1 and TLS Certificates
As you probably know, SAML endpoints in metadata are protected with TLS. In particular, browser-facing endpoints in metadata are associated with TLS certificates signed by a trusted CA (i.e., a CA trusted by your browser). Browser-facing TLS certificates do not appear in metadata and are therefore out of scope with respect to the Implementation Plan.
Not surprisingly, the signature over a trusted (browser-facing) TLS certificate typically relies on the SHA-1 digest algorithm, and so the CA and browser vendors are also migrating to the SHA-2 family of digest algorithms. This will be the topic of a future blog post.
SHA-1 and SAML Certificates
The rest of this section concentrates on the public keys bound to certificates in SAML metadata.
A SAML entity is a secure web server that produces and/or consumes SAML messages. These SAML messages may be signed or encrypted using asymmetric crypto operations. A relying party uses a trusted public key in metadata to verify an XML signature or perform an XML encryption operation. These public keys are bound to long-lived, self-signed certificates in metadata. Like all X.509 certificates, the signature on a certificate in metadata uses a hashing algorithm, which is almost always the SHA-1 hashing algorithm.
Fortunately, there are absolutely no security considerations concerning the signature on certificates in metadata. In fact, the only concern is the actual public key bound to the certificate in metadata. Every other aspect of the certificate is ignored by the SAML software (or at least software that’s doing the Right Thing). That’s why InCommon recommends the use of long-lived, self-signed certificates in metadata: the focus is on the key, not the certificate.
Don't fiddle with the signature on self-signed certificates in metadata. Until the signature on a certificate in metadata actually becomes an interoperability issue, it is best to leave it alone.
SHA-1 and SAML Assertions
Like metadata, SAML messages may be digitally signed for authenticity and integrity. Any SAML entity (IdP or SP) may sign a message but in practice it is the IdP that wields a signing key since SAML assertions issued by the IdP MUST be signed. The SP uses the IdP’s public signing key in metadata to verify the signature on the assertion just-in-time, that is, when the assertion is presented to the SP by the browser user.
It is believed that most IdPs in the InCommon Federation are signing assertions using the SHA-1 digest algorithm. This is because most IdP deployments use the Shibboleth IdP software, which is known to support SHA-1 only, at least out of the box.
SAML deployers should take note of the following facts:
- If your SAML deployment can consume metadata signed using the SHA-256 digest algorithm, it very likely can consume SAML assertions signed using the SHA-256 digest algorithm.
- The ability to sign SAML assertions using the SHA-256 digest algorithm is not fully supported by the Shibboleth IdP software (but please read on).
Clearly the former is an enabler while the latter is an inhibitor. Without ample software support, InCommon can not realistically recommend a firm SHA-256 migration path for IdPs. The best we can do is to ensure ubiquitous SHA-256 support at the SP and then encourage individual IdPs to devise a migration plan that makes sense for them.
After June 30, 2014, almost all SPs in the InCommon Federation will support SHA-256. Given this, each IdP should decide, subject to its own unique set of circumstances, when to begin signing SAML assertions using the SHA-256 digest algorithm.
SimpleSAMLphp has good support for signing assertions using the SHA-256 digest algorithm. It can be configured to sign assertions on a per-SP basis, that is, a simpleSAMLphp IdP can sign assertions using the SHA-1 digest algorithm for some SPs and the SHA-256 digest algorithm for others. Therefore sites running the simpleSAMLphp IdP software can and should migrate to SHA-256 as soon as possible. A perfectly reasonable strategy would be to configure the use of SHA-256 by default but to fall back to SHA-1 for those few remaining SPs (perhaps external to InCommon) unable to handle SHA-256.
To enable SHA-2 support in the Shibboleth IdP, you can use a 3rd-party extension to enable SHA-2 capability for Shibboleth IdP versions 2.3 and later. Note that the Shibboleth configuration is all or none, either SHA-1 or SHA-256, so Shibboleth IdPs are advised to wait until after June 30, 2014, when all SPs will be consuming metadata signed using a SHA-256 digest algorithm.
Actually, it’s a bit more complicated than that for Shibboleth deployers. Shibboleth IdP v3 is due out later this calendar year and of course v3 will fully support the SHA-2 family of digest algorithms, including SHA-256. So the question for each Shibboleth deployer is whether to install the extension in their current deployment of Shibboleth IdP v2 or wait for Shibboleth IdP v3?
For those IdPs certified or looking to be certified in the Assurance Program, the Bronze and Silver Profiles require that "Cryptographic operations shall use Approved Algorithms" and since SHA-1 is no longer an "Approved Algorithm," it can not be used to meet the requirements of Bronze or Silver. However, the InCommon Assurance Program is working on releasing a variance to the requirements (called an Alternative Means) to permit the use of SHA-1 until January 15, 2015.
Be advised that if you intend to interoperate with a federal agency SP that requires Bronze or Silver, assertions MUST be signed using a SHA-2 digest algorithm. Such an IdP should begin the migration to SHA-256 immediately.
Phasing Out the Legacy Metadata Aggregate
On March 29, 2014, the legacy metadata aggregate at location
will be replaced with a redirect to the following new location:
All deployers are advised to migrate to one of the new metadata aggregates ASAP but no later than March 29, 2014.
In recent weeks, a number of people have asked me: What will happen if I do nothing? One answer is:
In more than 90% of the cases, if you do nothing, your deployment will continue to function as normal after March 29th.
We could drill down on that other 10% but consider this: We know with 100% certainty that a redirect will be installed on March 29th, so knowing nothing else but that simple fact, we can conclude that all deployers are better off migrating to the new fallback aggregate than they are doing nothing because all other things being equal (which they are) a controlled migration is always safer than a forced migration.
If you’re running Shibboleth, migrating to the new fallback aggregate is as simple as changing the URL in your Shibboleth Metadata Config. Go ahead, schedule that simple configuration change subject to whatever change management policy you have in place. You’ll know in a few moments if it’s going to work, and honestly, there is a very high probability it will just work. If it does, you’re home free because you can complete the rest of the migration on your own time. If it doesn’t work, you can quickly back out the change and invoke Plan B.
So why does that simple config change work without bootstrapping an authentic copy of the new metadata signing certificate? Because Shibboleth ignores all the certificate details except the public key bound to the certificate, and that key hasn’t changed, so we're good to go.
For more detailed information, consult the Metadata Migration Process wiki page.
Questions? Join this mailing list: https://lists.incommon.org/sympa/info/metadata-support