QA for SaaS products

QA needs coffee

QA effectively applied to a SaaS product


QA needs coffeeI wish I could say that, when given the assignment of auditing an HR recruiting system delivered as a SaaS (Software as a Service) product I could have predicted exactly what path we’d have to go down to get fixes made. I could not have been more wrong.

For many years I worked on the vendor side. We owned the code and we fixed it. This process is well known in software circles.

In SaaS you subscribe to software and you are the customer. The vendor still owns the code, but they are on the opposite side of the development fence.

When working for the Commonwealth of Massachusetts to assess the Oracle product Taleo, which we were going to use to update and modernize the hiring process, I invented the following process to get accessibility bugs fixed using trial and error as my guides.

First, I used a keyboard-only approach to audit the two initial sections of the HR recruiting system. Then I used the WAVE toolbar and the WAT toolbar to look into code violations. Then used both JAWS screenreader and ZoomText to replicate the user experience with assistive technology.

The candidate portal, where John and Jane Doe apply for a job, was written in HTML. There were bugs and issues, but they could be fixed. The hiring manager portal was another story. Written in a version of Flash called Flex it had no redeemable value for use with keyboard or JAWS screen reader.

Another day I’ll write about how we dealt with the hiring manager portal. This is to explain how we managed to get 20 accessibility fixes into the last two releases of the Taleo product.

As customers we had access to an externally facing web application where we could log bugs. In a four month period I logged 30 tickets, which represented 42 separate bugs (some were consolidated because they were similar) and worked with 17 different customer service agents (CSA) in the US, Romania and China.

The process of getting bugs accepted by the CSAs and moved to the next step, which is to place them in the internal development bug tracking system was painful and slow. Each CSA had to understand enough of both AT and the accessibility guidelines to believe that the issue was really an issue.

If they didn’t accept it as a bug then their default is to politely reject the bug and call it an “enhancement”. During the time I worked on this project Oracle opened a second website for enhancements where customers were encouraged to log their wish lists and try to socially promote them. I really hated this “popularity contest” approach but we had to use it.

We engaged in a series of cross functional meetings between the Commonwealth and Oracle representatives from sales, product management and customer service. We explained exactly what barriers a person using JAWS would face when trying to navigate the original site. In this way several “enhancements” were accepted into the faster track “development fixes.”

Of the original bugs filed, 21 were included in the past two releases with 8 more pending for the future. Although it seemed at times like I was doing the QA for Oracle on this product I really didn’t mind because I wanted to get the Commonwealth and other Taleo users the most accessible product possible.

Accessibility jobs

subway to work

Who is working in Accessibility now?

subway to workIt was intriguing to read Webaim’s July,  2014 survey results of accessility practioners. Vast oversimplification: 900 respondents, 90% of respondents work in US or Europe, 1/2 of them work in accessibility as a “primary” part of their jobs, however @ 60% of them work on accessibility less than 20 hours a week. Only 22% said they were developers and only ~4% said they were QA professionals. These numbers will get interesting later in this article. Median salaries were between $60k and $80k but fluctated widely depending on amount of time spent in accessibility, education levels, and industry (government/ corporation/ consultancy).

Reading (and participating) in the survey started me wondering if the field is getting more exposure. Are there really more jobs? What industries are looking to pay salaries to people with accessibility related skills?

So I set out to answer some basic questions. How many jobs are there in Accessibility? What kinds of skills are employers looking for? What is the growth potential in the market?

Gathering stats on accessibility jobs

Query on on October 25, 2015 using “%Accessibility%” in the title. No limits on location other than the United States. There were 107 results, some of which were redundant. Multiple recruiters were hawking the same position. Culling for duplicates I found 80 jobs. They are heavily weighted to IT (half were for developers or QA professionals), almost ¼ were management or administrative positions and the final ¼ were a mix of electronics, UI/UX, technical workers.

When I searched more broadly within using just the term ‘accessibility’ to pick up any jobs using the word in their descriptions I found 11,442 jobs. Unfortunately, many of these job descriptions include a phrase like “if you need accessibility accommodations to apply for this job, please contact <email and or phone number>” In other words, the jobs are NOT about accessibility.  Other off track uses of “accessibility” or words that stem from it are: “Access” databases, job includes enticing ”access” to areas of beaches, skiing or other amenities or job duties include “providing access to systems for onboarding new employees”.

The jobs that truly include some skills or expectations of understanding about This confirms our finding of the 80 jobs on which used the word in the title. More and more development opportunities include a mention of accessibility, especially for front end and full stack developers. Even though it is still rare to see the proper skills taught in computer science classes, the subject might be mentioned in boot camps. Anecdotally, developers just run across a request to “make the site accessible”, leading to them doing whatever research they are capable of and/ or have time for.

Query on on October 25, 2015. Same criteria as above. Job title contains “%Accessibility%”= 58 results. Keyword contains “accessibility” = 5,303 results. Again, roughly half were development or QA jobs, about a quarter could be cast as management/ administrative and the last quarter didn’t fit neatly into any category, thus we’ll call them “other” for the moment. Again, most of the jobs in the larger data set either reference the employers willingness to provide specialized accessibility support for their candidates or include some other use of “access” which is not relevant to our search.

Skills requested

What are employers looking for? From job descriptions:


“If HTML, JavaScript and CSS make you want to get out of the bed in the morning, this is probably  a good job for you.” – SSB BART Group, San Francisco

“Experience preferred in WAI-ARIA, HTML, CSS, JavaScript, AngularJS (optional)” contract in Richmond, VA

QA testers:

“Screen readers, voice recognition, html/ CSS. Knowledge of industry standards and regulations (WCAG 2.0). Use of automated testing tools.” contract in Richmond, VA

“Be able to educate and guide engineers in best practices. Use and help improve existing processes.” Redmond, WA

Accessibility Lead:

“BA in HCI, Engineering or equivalent professional experience. Thorough knowledge of Accessibility guidelines. 5 years experience creating accessible products and using assistive technology. Three years native mobile experience with iOS/Android/ Win8/ web mobile expe

How many people have disabilities?

store signs

Statistics and the real story behind them

Census Bureau – 2012

shops-shopping-mall-signs“The U.S. Census Bureau released a report containing updated statistics on the population of people with disabilities in the U.S.  The Bureau reports that 56.7 million Americans (18.7% of the population) have some type of disability.

Some statistics from the report via Seyfarth Shaw’s very informative ADA Title III blog.

“The Bureau reports that among people age 15 and older:

8 million have a vision impairment

8 million have a hearing impairment

31 million have difficulty walking or climbing stairs, including 4 million people who use wheelchairs and 12 million people who use canes, crutches, or walkers

20 million have difficulty lifting or grasping.” 

Here’s another set of numbers to wrap your head around.

“More than 50 million Americans with disabilities – 18% of our population – are potential customers for businesses of all types across the United States.

“This group has $175 billion in discretionary spending power, according to the U.S. Department of Labor. That figure is more than twice the spending power of American teenagers and almost 18 times the spending power of the American “tweens” market.”

Accessibility attracts not only people with disabilities but also their families and friends. Like others, these customers often visit stores, restaurants, movie theaters, and other businesses accompanied by family or friends. This expands the potential market exponentially!

This market is growing fast. By the year 2030, 71.5 million Baby Boomers will be over the age of 65 and demanding products, services, and environments that address their age-related physical changes.”

There’s a lot of money to be made in getting this right.

What retail stores can do

Tiffiny Carlson, The Mobility Resource writes via the Huffington Post about small retail shops and grocery stores. She includes tips like leaving room in the aisles so that people using a wheelchair can navigate through. There are good points discussing staff training so that employees know when and how to offer help.

She also emphasizes that the store will have a customer for life if they are treated well. Since most people who have a disability have family and friends those people will also want to sign on as loyal customers.

“There are over 48.9 million living with a disability in the U.S.

Taking notice and appreciating our buying-power can be one of the best business moves you’ve made all year. Our population just in the U.S. spends $150 billion annually.”

Mart Carts available

An article on Disabled World shows that even a temporary disability (like waiting for knee surgery can have a profound effect on one’s day to day ability to get around.)

This article brings very familiar stores and organizations into the conversation (Costco, Goodwill, Wal-Mart, Ace Hardware) into the conversation about which outlets in her area had mart carts available. The little observations about which stores had diligently plugged in the ones needing recharging and how helpful the staff was gives life to the conversation.

Staff training

Just in case you thought your staff had a full 7 seconds to make a good first impression, guess again. Most recent research reveals that a person makes a positive or negative judgement within 1/10 of a second — and they don’t usually change their first impression. Your staff, especially your front line staff, needs training and coaching to apply simple, practical standards when they encounter a person with a disability. Pret a Manger restaurants worked with the Equal Rights Center (ERC) in 2012 to make sure that their facilities were accessible and their staff trained in disability etiquette.

Depending on your business this might involve simple transactions, like offering to read a menu out loud to a customer with low vision or writing notes back and forth for a deaf customer. Your marketing and customer service department can also develop additional materials in Braille or audio which can be presented as options in a friendly and supportive manner.

We provide training materials to sensitize and support your support staff. Contact us for more information.

A Customer is a Customer

A customer is a customer. But some customers are different. Some have special needs. Actually they all have special needs and since they are customers we want to know what they are.

When talking about demographics any successful business person knows that it is important to identify what motivates and encourages their customers. Interest in your industry (sports fan for a sports related product, for instance) make it more likely they will buy.

Nearly a fifth of Americans have some kind of disability, and they have an annual discretionary income of $200 billion. As spendthrift baby-boomers enter their golden years, these numbers are expected to rise.” – the Economist – April 25, 2015

Let’s back that up a bit – $200 Billlion!! We all want as large a piece of that market as we can get. JPMorgan Chase certainly does.They are in the hiring process of a Managing Director, calling the title Head of Accessibility Affairs. This person will consolidate and lead the $2.6 trillion company in the area of Accessibility.

A fascinating, first hand account of shopping by a woman who is temporarily experiencing the need to use a mart cart while awaiting knee surgery.