Why this matters
Good QA programs do more than testing whether the developers understood the assignment.
QA testing is a collaborative role requiring empathy for people using the product and a shared goal of ensuring a good user experience.
Team commitment KPIs
What requirements must the team agree to meet if your organization is going to be successful?
Testers need to be capable of manually testing and describing the problems they find, not just running automated scans and listing WCAG criterion failures.
What’s covered
Can explain why accessibility is a requirement
Team members can possess varying levels of technical understanding, but all should be alert to the basic reasons accessibility is a requirement.
Accessibility target policies
At a minimum, any team needs to understand accessibility targets exist, are defined by enterprise policy and enforced by leadership.
This bare minimum adoption achieves some level of compliance, even when it’s begrudging.
However, if the reasons for accessibility policies are taken to heart in service of the customer, products can dramatically improve.
Living your organization’s values
Accessibility isn’t just the right thing to do, it’s the smartest thing to do.
Every organization has a set of values, often including core ethical tenets like treating people with respect and doing the right thing.
How does accessibility fit those values? How does ignoring accessibility breach them?
A tool for innovation
Accessible design and development builds better products for everyone. When teams put accessibility at the beginning of their processes, they create more valuable products for your enterprise.
Competitive advantage
26% of the US population has a disability that requires accommodation, making people with disabilities the largest minority in the United States. This adds up to billions of dollars in combined purchasing power.
Avoiding legal risk
Accessibility is the law. Designing and building accessibility into products also helps the enterprise avoid legal risk and liability due to a customer complaint.
Can characterize automated and manual testing
Manual testing
Manual testing is precisely that: a human actually testing the experience using the screenreader and browser combinations you need to support.
Experts can deliver an organized report of defects by severity. This is a necessary tool for improving the customer experience.
Limits of manual testing
A manual test isn’t the same as a usability study, but it is effective in uncovering the issues your customers experience.
Manual testing is performed by people, and perception of what constitutes a defect can vary slightly from one tester to the next. It will be helpful for your testers to reference your severity definitions, and use uniform testing acceptance criteria.
Automated testing
Automated tests find programmatic errors, but can’t describe actual customer experiences. Just like a spell checker, automation can flag non-issues while missing legitimate problems because it can’t understand context and intention.
How to use automated scans effectively
Scanning tools quickly pinpoint syntax defects in code. Some flagged issues won’t affect the customer experience, but you should exercise scrutiny and manual testing if a web page is riddled with invalid code and errors.
Limitations of automated scans
Testing tools have value. But it’s important to understand their drawbacks. Even the most robust tools can identify less than half of the potential defects on a page.
Code can be inaccessible for a person using their keyboard or screen reader without being flagged as invalid markup by an automated tools.
Practical examples
- Automated scans can instantaneously test checkboxes for properly associated labels and other code attributes, but can’t tell you if the labels make sense.
- Automation tools can flag an image for missing alt text, but can’t tell you if it would be better for the screen reader to ignore a particular decorative icon.
- Custom components, like an accordion expander, could be inaccessible with the keyboard and yet be formed of valid code that won’t raise an error.
Can effectively use assistive technology
Testers need a solid understanding of how to use assistive technology and what expectations to have from components.
When testers don’t understand how to use the keyboard or screenreader on desktop and mobile, they will create invalid defect tickets. This will waste time as developers review and may even try to solve a non-existent problem, likely creating a real problem.
Can understand atomic accessibility acceptance criteria
Acceptance criteria have to be specific enough to cover core functionality, including every quirk or difference between the five screen reader and browser combinations, but broad enough to not become overly verbose. AtomicA11y.com is one way to generate acceptance criteria.
What are the components of acceptance criteria
Name
This is how the element’s purpose will be read to the screen reader. For example, the name of a link is typically the inner text of the link. The name of an input is typically the label like, “First name”.
Role
Every element has a role. A radio button’s role is “radio”. A submit button’s role is “button.”
State
Many controls have a state. For instance, a checkbox input can be “checked” or “unchecked.” A toggle switch can be “on” or “off”.
Group
Nearly all components work as part of a bigger context or group of elements.
For instance, a collection of radio inputs need a group name. Headings should exist in an structured and logical pattern, starting with an H1, (typically the page content title), with major sections using an H2 at the beginning.
Every interactive element can have all of these criteria. Non-interactive elements will vary. For instance, headings have no state property.
Can interpret accessibility assessments
Teams will need help and training on how to interpret, prioritize and act on information from full assessments. The content is often based around WCAG criteria and may or may not offer techniques for remediation.
Definition of ready requires atomic accessibility acceptance criteria
Any QA process defines what is required for testing to begin. This is called a “definition of ready.” Without a definition of ready a team cannot estimate the amount of work required.
Acceptance criteria from MagentaA11y can used to clearly define test instructions.
Assistive technology test suite
Prioritize your test suite by what devices and browser combinations your customers are using.
Setup of these tools can vary. For instance, if your team is already using Macs then they already have VoiceOver and can install NVDA using a virtual machine environment, without having to set up a separate physical PC.
Desktop
Keyboard
Successful keyboard interaction is a prerequisite for testing with a screen reader.
PC + NVDA + Chrome
If you’re only going to test with one screen reader, it should be NVDA. It is free and it is very demanding of compliant code.
Mac + VoiceOver + Safari
If you’re testing with two screen readers, VoiceOver should be the second. VoiceOver is built into MacOS and pairs with Safari.
PC + JAWS + Chrome
Most of your customers with vision disabilities in the U.S. will be using JAWS because it’s subsidized by the federal government. JAWS is more forgiving of non-compliant code than NVDA or VoiceOver, so despite its market share, it is not always ideal as your sole testing platform.
Mobile
iOS device + VoiceOver + Bluetooth keyboard
VoiceOver is built into iOS and pairs with Safari.
Android device + Talkback + Bluetooth keyboard
Talkback is a free screen reader for Android and pairs with Chrome.
Uniform automated testing tools
There are a multitude of free automated testing tools available. Simplify compliance by using one uniform tool used for development, QA testing and pipeline gating.