Why this matters
What can effectively be reported with data
Assessments and automated reports expressed by severity in each product.
Use data precisely to tell a story
Sheer numbers are easily overwhelming, and not always useful to express the scope of the problem.
Understand its limits
The data available are only indicators of compliance or non-compliance. A product could have many relatively inconsequential issues detected by automated tools with few issues that actually affect customer experience.
While you shouldn’t plan your entire program from casual benchmarking, you need to understand peer organizations and competitor efforts. This can help you make the case for a stronger accessibility program if you’re behind.
Compare accessibility program against peers
It can be helpful to gauge your enterprise’s commitment to accessibility by understanding what peers or competitors are doing.
If you find that your competitors are placing significant resources into accessibility innovation and compliance, that’s a helpful marker to reach.
Compare compliance against peers
Run an automated assessment on peers and competitors. For example, an automated testing tool could produce an assessment on a competitor’s public facing website with little commitment.
Spend time learning what reports are valuable to leadership. For example, leadership may prefer to point to a single number or average score. In other contexts, leaders may wish to dive deeper to drive specific results.
Quantity of defects by severity
Categorize defects by severity to add meaning to trends. It can be difficult to describe progress on a large number of defects. But, if your reports can show that high severity issues have been halved, you’ll present a more digestible concept for leadership.
Remediated vs net new defects
Unfortunately, it’s entirely possible for teams to remediate defects from an assessment, while simultaneously creating new issues when practices don’t change.
Maturity model compliance
Teams consistently following best practices should achieve positive accessibility outcomes. For each role, team members and managers should be able to report compliance with each best practice.
Reports for product teams
Individual product teams will generally be responsible for remediating their own defects.
Set the right expectations
Don’t deliver assessment reports and wait to see results.
Set the expectation that teams will require a degree of help from accessibility coaches to understand and act on the reports. It’s a serious red flag when teams don’t reach out for help. They are either deprioritizing the work or overestimating their team’s ability to remediate.
Defects from an assessment
Automated scans often produce an overwhelming number of defects, and manual assessments may point out issues the team simply isn’t equipped to understand.
Time will need to be spent onboarding the team to the assessment process, what the expectations and deadlines are for remediation, and where to find help.
Validation check of remediation
Unfortunately, it’s entirely possible for teams to believe, or simply claim, they have remediated defects when that is not the case. In the interests of due diligence, remediation work should be manually validated, or at least spot checked, for quality.