Accessibility Testing: Establishing A Screen Reader Test Plan.

What are screen readers?

Screen readers are a text-to-speech software that work on top of a web browser (and other applications) to read screen content out to users who have severe sight problems, reading problems or learning disabilities. There are a multitude of options out there on the market with users choosing their screen reader dependent on it’s capabilities, cost, whether it is supported by their employer or school and so on.

‍

The current problem

Testing with screen readers when developing web content is key but can be tricky given the diversity between screen reader capabilities and support of existing and new technologies such as Flash, PDF, WAI ARIA, SVG and, new developments in HTML (formally known as HTML5). Throw into the mix that software upgrades can vary considerably and you find yourself not knowing if content not being read correctly on a page is to do with your content, a bug in the reader or a quirk with the browser.

‍

Available screen readers

Popular paid for screen readers, often supported by employers and schools are Jaws for Windows, WindowEyes and Hal. Free screen readers include VoiceOver on Mac (as well as iPhone and iPad), Orca on Linux, Non Visual Desktop Access (NVDA) and System Access to Go (SAToGo). NVDA, Orca and SAToGo are also open source. See a comparison of screen readers on Wikipedia for a more comprehensive list of what screen reader is supported across platforms and their capabilities.

‍

1 – Test with actual screen reader users

While testing our own content is great, especially when trying out new coding techniques, nothing is as good as feedback from real users. There are organisations out there who can carry out screenreader testing for you but don’t think that if you can’t afford it you can’t test with real users full stop. You may have screen reader users in your organisation or wider network of contacts who are willing to spare five minutes to give informal feedback. You can also ask in forums such as Accessify or over Twitter for feedback and support. The free online book from Shawn Henry-Lawton (Just Ask – Integrating accessibility throughout design) has some useful tips on involving people with disabilities in your project.
If you involve screen reader users formally make sure you are clear about what screen readers you want to test with (more of that later), what browsers and have a clear set of tasks for testers to follow.

‍

2 – When to test

As with all testing you want to start as early as possible and do it as often as possible. Having said that basic pages built with semantic HTML should just work for screenreaders so the need to test these is less urgent unless you are familiarising yourself with how screen readers work.
If using progressive enhancement you will need to test if the screen reader user can access core content and functionality. Visually a sighted user may easily understand a tab panel within a page but a screen reader will literally communicate it as HTML links, unordered lists and so on unless WAI ARIA has been used. To ensure that ARIA implementations work always test.
Other areas that you should test may include how your heading hierarchy works not just in pages but across the site as well as keyboard access.
You may also want to get overall feedback on specific areas of accessibility such as how logical the heading structure is across the site (not just the pages) and how well ARIA roles and properties describe interaction with the site.
If you involve screen reader users formally make sure you are clear about what screen readers you want to test with (more of that later), what browsers and have a clear set of tasks for testers to follow.

‍

3 – Invest in training

Screen reader skills can be hard to learn and are often championed by one person in the team. This can act as a weak link in the chain should that person be unavailable or move on. Often the person testing is also the person developing and while this can have some benefits (when testing new solutions for example) this can have its drawbacks. Instead of having just one person with the skills to test using screen readers have them document a plan and maintain a plan and set up training courses. Ideally getting a real user or Consultant to come in and spend time with the team is best.

‍

4 – Decide your screen reader test list

As well as majority screen readers such as Jaws and WindowEyes also test with minority screen readers such as NVDA and VoiceOver. Some readers have better support of newer technologies than others. NVDA has advanced support for WAI ARIA for example, and has the added benefit it’s free whereas Jaws 12 (which also has good WAI ARIA support) is not. VoiceOver is also an interesting choice. While it may not be hugely popular just yet there is anecdotal evidence in WebAim’s screen reader survey’s that it’s popularity is on the rise.
I can only see this increasing as users want to have a seamless experience across devices using iPads and iPhone 4 alongside their Macs.
Screen reader software updates are released fairly frequently so your list of approved screen readers will need to be reviewed and amended along with major new releases. Underpinning all of the above is ensuring that you never build for a specific screen reader or version. Access technology is not perfect and has quirks, while it is good to be aware of this and create support where possible you should develop using guidelines and web standards appropriately.

‍

5 – Browsers

Keep abreast of browser releases and beta versions. While testing on a beta version may not be appropriate as it is subject to change it’s worth noting key updates that may influence screen reader access. Typically this could be added support for new technologies such as WAI ARIA and new enhancements in HTML (formally HTML5). Note however that browsers tend to add support for web technologies quicker than the more established and paid for screen readers such as WindowEyes and Jaws. Smaller, free screen readers (who arguably have less constraints aside from resource) can be more agile in development.
The Paciello Group blog is a great resource to learn about WAI ARIA support in browsers.

‍

6 – Don’t screen reader test in isolation

Never do a screen reader audit alone or you’ll end up in a black hole of potentially making your site technically accessible to screen readers but possibly to the detriment of other users of all abilities. An overall accessibility plan should include expert and user testing, code reviews and validation.

‍