A new flavor of QA job order is rolling into recruiters nationwide. This is a request for an immediate, experienced QA professional who does accessibility testing. Here are timely suggestions for what the recruiter needs to ask of:
- The employer at time of receiving the job order
- The candidate at time of initial screening/ interview
First: a short background and list of some terms the recruiter needs to get up to speed on the topic. What, homework!? Yes, homework, so you’ll do your job right.
Even though the ADA (Americans with Disabilities Act) is 25 years old its application to software is still being defined. In short, this means that you are not going to find an exact definition anywhere of what the exact expectation of success is in this area.
With that said, gain some passing understanding of what these two terms mean: “Section 508” and “WCAG 2.0”.
Section 508 is a set of standards the US Federal Government has established to measure the Electronic and IT hardware and software it uses against. Please note that these standards were last updated in 2000 (which is a lifetime ago in internet terms.) A “refresh” of the standards is due soon.
This is where WCAG comes in. While the Section 508 has worked its way through the government process at a snail’s pace there are others in the world working on similar guidelines. Not a law, nevertheless most software companies look to align with these guidelines because they are more comprehensive and up to date. NOTE: Do not try to read most of WCAG. There are thousands of pages and as many ways to implement the guidelines as there are developers and planners.
5 myths:
Myth #1
Screen Reader (JAWS, NVDA) equals a “test tool”.
Reality Check:
Many employers will ask that the QA skill set includes “testing” with JAWS or NVDA. Do not fall into the trap of thinking that these are test tools.
These software programs are Assistive Technology. They help people who have trouble accessing the internet or such programs as Microsoft Word convert the information encoded on the screen into audio or refreshable Braille outputs.
When testing for accessibility, qualified and aware QA professionals typically use screen readers to validate the end user experience of websites or software programs. This is hugely helpful, but is not the main method of testing.
Recruiter Question to Employer: Are there specific screen readers your users use? In the case of employee used software they probably have a preference. In cases where the software to be tested is for customers you will have to use best practices, which general means JAWS for US based companies. NVDA is also widely used.
Recruiter Question to prospective QA Employee:
What test tools do you use to validate the code against accessibility standards?
[Give points for answers such as WAT (IE toolbar from The Paciello Group http://www.paciellogroup.com or WAVE toolbar (Firefox and Chrome plugins from Webaim http://webaim.org).
Give points for answers which discuss evaluating the code for items like form labels, WAI-ARIA roles and properties]
If the QA person says they use JAWS, for instance, to test with, ask them how they do that.
Myth #2
User Experience is the same for all users.
Reality Check:
Even if a site is usable for the general public there is a need to do additional usability tests for people with disabilities (PWD)
Real example: the code was properly designed to be read by a screen reader, but because the fields in the design of the screen were set up in a certain order the screen failed the usability test.
A resume upload page on a very common HR recruitment site had an input box below the Submit button. The sighted user could easily see the Comments box and choose to type something like “Portfolio” or “Cover letter”. The blind, screen reader user would not encounter that box until AFTER they had submitted their resume. At this point there was no chance to add the comment or go back.
The page passed the Section 508 and WCAG criteria but failed the simple usability test.
Recruiter Question to Employer: Will there be an opportunity in this role for the QA person to discuss the site with designers and/ or product managers?
[Give points whenever an employer understands that this discipline is a cross-functional, cross-team effort.]
Recruiter Question to prospective QA Employee: What experience do you have with creating/ testing against User personas? How much experience do you have working with people who need assistive technology (AT)?
[Give points for any recognition of what UX is, what personas are and, most of all, if the QA resource has worked with PWD.
Give points if the QA resource has had any experience explaining that websites/ other software may present challenges to those who are using AT. If they have done so, ask them to describe the outcomes. ]
Myth #3
You have to test all pages on a site to do an accurate accessibility test.
Reality Check:
Given that so many sites have hundreds, or even thousands of pages, it is impractical to comprehensively test them.
The usual method is to concentrate on the primary pages which handle the most user traffic. (Home, Search results, Contact us, main shopping and check out pages, etc.) If those pages reveal serious problems they must be fixed first. Only later should the testing go deeper. Additionally, newly coded and added pages should include accessibility testing as they are prepared for production.
Recruiter Question to Employer: Will the QA resource be presented with the core workflows so they can include them in the testing? Who has developed the initial test plans?
[Give points if Product Management is involved]
Recruiter Question to prospective QA Employee: How many scenarios do you usually test? What are the main paths? Do you find that pages deeper in a site present different challenges than the Home page?
[Give points for any answers that indicate that the QA resource is familiar with accessibility being disproportionately built into the Home page and ignored further into the site. Give points to a QA resource who mentions that some features, while not on every page would warrant special testing – forms, videos (for captioning and general keyboard accessibility of video player controls)
Myth #4
Anyone who has a degree in Computer Science would know how to test for Accessibility.
Reality Check:
Fact is that most of the computer science programs, boot camps, and assorted free and for-pay courses do not even touch on accessibility related coding. Most of the programmers who know how to prepare their code for consumption by assistive technology and people who use alternative methods of understanding the UI have learned it “on the job” or by doing research on their own.
Recruiter Question to Employer: What specific skills are you looking for in a QA resource to test your site/ software for accessibility?
[Give points for any answer that acknowledges that most resources will not have gained this knowledge in a common computer science program. Give points if the company already has some knowledgeable people on the ground who can help the new QA resource.]
Recruiter Question to prospective QA Employee: Where did you learn about accessibility and accessibility testing?
[Be prepared to hear a long and drawn out version of their discovery process. Be patient and take notes. Do not believe them if they say that they easily discovered it in a class or another job.]
Myth #5
There is no need to do manual testing, because all the tests can be automated
Reality Check:
While there is much merit to automating as many checks against clean and correct code bases, even the most progressive automated testing tools can only check code violations (HTML mis-matches, for instance).
None of them can accurately replicate either the user experience for a person using keyboard only approach or various assistive technologies: See Myths #1 and 2.
Recruiter Question to Employer: Are you expecting to have the QA resource convert some of their tests to an automated system? If so, which one are you using and are you aware of any necessary accessibility plug-ins that will be required?
(Give points if they already understand that there is a limit to the usefulness of applying automated testing across the board to this effort. Part of your role here is to help set expectations of what is possible in reality.)
Recruiter Question to prospective QA Employee: Have you ever committed your accessibility tests to an automated process? If so, which one? How effective has this been? What do you know about accessibility plug-ins for the major automated test systems?
(Give points for honesty when the tester has not been able to do much useful with automated tools. Give big points if they mention anything covered in this article by CogApp or this one where researchers in Norway examined the results of 12 automated accessibility checkers. Yes, done in Norway, but the checkers are global.)
Wrapping it all up:
QA can and should do testing re accessibility diligently, deliberately and determinately whenever a software project is undertaken in 2016. However, that QA is by default dramatically different from other areas of QA.
Get familiar with the ins and outs and prepare to make significant money for your recruiting firm and your QA resources. This is a niche that is not going away. Think about placing QA security testers and prepare to reap the benefits in the same kinds of ways.
I wish I could say that, when given the assignment of auditing an HR recruiting system delivered as a SaaS (Software as a Service) product I could have predicted exactly what path we’d have to go down to get fixes made. I could not have been more wrong.
No, it’s not a menu item, digital or restaurant! A VPAT stands for Voluntary Product Accessibility Template. This is a form based on Section 508 accessibility compliance (or lack thereof) for a software or hardware product. Originally the form and the legislation was designed for federal contractors. Over the past 20 years or more Section 508 has been informally recognized as a standard for accessibility all should follow.
As soon as your content includes videos you should be thinking about captioning. In the simplest form, captioning is those words on the screen that are readable as the audio portion of the program is spoken. The idea is that captioning is only useful for deaf or people who are hard of hearing. Not true. An oft