At the November TAC F2F, we discussed having a matrix of best practices by which to evaluate registered sites to help set expectations and create peer pressure. This is a preliminary set of suggested criteria.
The Best Practice Matrix might also be thought of as the "Community Wall of (Fame | Shame)." Emphasizing "Fame" or "Shame" would be a conscious decision on our part. It seems both approaches would be appropriate and effective in practice. For example, the IDs of entities requesting the unsupported WAYF might be published on the "Wall of Shame" while a list of those entities supporting SAML V2.0 would be published on the "Wall of Fame."
Policy / Non-Technical
- POP Available
- Security Incident Contact Registered
- Does this also imply adherence to the recommended incident response process?
...
- Accessing the (unsupported) InCommon WAYF
- Deploying an Unsupported Version of Shibboleth
- Expired Certificates in Metadata
Deployment Practices
- SAML 2.0 Support
- IdPs with TLS-protected HTTP-Redirect SSO
- SPs with TLS-protected HTTP-POST ACS and an encryption key
- Support for SAML 2.0 persistent NameIDs or eduPersonTargetedID
- Perhaps support for other attributes are worth noting?
- Full saml2int conformance
- Consent-based support for particular attributes (i.e., no admin involvement needed)
- Keys of less than a certain age
- We should consider what, if any, age is actually "too old"
- Appropriate error pages
- Perhaps subjective, but I'd start with having actual contact info for users and a reasonable indication of what to do, maybe not using the Shibboleth logo?
...