At the November TAC F2F, we discussed having a matrix of best practices by which to evaluate registered sites to help set expectations and create peer pressure. This is a preliminary set of suggested criteria.

Policy / Non-Technical

• POP Available
• Security Incident Contact Registered
  • Does this also imply adherence to the recommended incident response process?

Deployment Practices

• SAML 2.0 Support
  • IdPs with TLS-protected HTTP-Redirect SSO
  • SPs with TLS-protected HTTP-POST ACS and an encryption key
• Support for SAML 2.0 persistent NameIDs or eduPersonTargetedID
  • Perhaps support for other attributes are worth noting?
• Full saml2int conformance
• Consent-based support for particular attributes (i.e., no admin involvement needed)
• Keys of less than a certain age
  • We should consider what, if any, age is actually "too old"
• Appropriate error pages
  • Perhaps subjective, but I'd start with having actual contact info for users and a reasonable indication of what to do, maybe not using the Shibboleth logo?

Implementation Support

• InCommon Implementation Profile conformance
  • Could call out Metadata IOP as a subset, but my guess is few products would support that without the rest
  • Could identify "exceptions to conformance" to highlight specific missing capabilities or could break profile into separate features in the matrix