The business case for issue prevention: Extreme Accessibility

I originally posted this on LinkedIn. It turns out that LinkedIn is a poor place for me to reach my audience, as my average post gets 10x as many views as this post first got on LinkedIn.

There have long been discussions on the accessibility industry surrounding the business case for accessibility. The Education and Outreach Working Group at the Web Accessibility Initiative of the W3C developed a large set of resources on this topic to which I posted a series of blog posts in response. During that series, I concluded that “Reduction of Legal Risk” had the strongest ROI. At the time, however, I failed to realize what so many others also continue to overlook: accessibility issues are bugs and should be treated that way. Accessibility is unique in that it exists at the crossroads of technical quality and user experience. On the technical side of things accessibility can be seen as a quality domain. Bugs in the code result in user experience issues for specific populations of users. Resolving the bugs increases quality.

It is with this sentiment in mind that gave rise to Tenon‘s mission statement:

To create robust, accurate, forward-thinking software that integrates accessibility testing into all phases of development and workflow. Together we can achieve a more accessible Web.

We meet this mission by providing an excellent service and ecosystem to allow our customers to prevent bugs before they happen.

Remediation takes money and time. Prevention saves money and time

It almost seems too obvious to mention: no bugs means no money spent on fixing bugs. But bugs are inevitable. Given the inevitability of bugs, when the bugs are found and fixed is the critical factor in saving money. How much money is difficult to determine. Back before the web, Barry Boehm estimated the expense of post-production remediation to be 100x the cost earlier in the software development process. This became known as the Boehm Curve. While the “100x” estimation has been questioned by many, what isn’t under question is that late discovery of defects is orders-of-magnitude more costly. The cost of defects is massive.

…although all errors cannot be removed, more than a third of these costs, or an estimated $22.2 billion, could be eliminated by an improved testing infrastructure that enables earlier and more effective identification and removal of software defects. These are the savings associated with finding an increased percentage (but not 100 percent) of errors closer to the development stages in which they are introduced. Currently, over half of all errors are not found until “downstream” in the development process or during post-sale software use. (
Source: NIST)

Defects don’t just cost the money needed to fix them. Simple metrics like raw cost to fix a defect only take into consideration the time it takes for a developer to fix a defect. The economic benefits of avoiding defects extend into and beyond all phases of development including the maintenance phase. In total, the Cost of Quality is the Cost of Conformance + Cost of Non-Conformance and that includes typical development, remediation, maintenance, and user support costs. Many thought leaders in the quality domain focus on these costs alone. Others, such as Capers Jones agree that simple cost-per-defect metrics aren’t quite enough and apply other metrics such as function points.

Beyond development and remediation, there other costs you may be unaware of such as Costs of Acquisition and (missed) Customer Lifetime value. These are metrics intrinsically tied to a business’s profitability. System defects impact these factors as well for cases where the defects impact core system tasks or features used during conversion. Something as simple as an inaccessible “Sign Up” button may completely prevent somewhere from 7-12% of visitors from converting.

Reliable web accessibility business case numbers are hard to come by due to the strong relationships between quality, usability, and accessibility, but it isn’t hard to imagine that the end result is the same, from an ROI perspective, as in the brick & mortar world:

three quarters (75%) of disabled people and their families said that they had done this (75% of disabled people and 76% of parents or carers). Looking at this across impairment, the percentage of people who have left a business can reach 83%. In particular, 83% of those with memory impairment, 81% of those with a behavioural impairment, 81% of people with autism and 79% of those with a learning disability said they had left a business for this reason. (
Source: Business Disability Forum)

Given the significant risk to project budgets and revenue from users, proactively addressing accessibility makes the most sense

The above image shows an annotated version of the famous Boehm curve (Source: Scott Ambler) showing that even in Agile projects, due to the length of the feedback cycle and the increased likelihood that effectively addressing the issue becomes more difficult: “The average cost rises exponentially the longer that you wait because you continue to build upon a shaky foundation”. Nowhere is this more true than in the case of accessibility, where design and platform decisions made early will impact production and issue remediation later.

Extreme Accessibility delivers significant ROI compared to traditional web accessibility approaches

Agile techniques, such as Test Driven Design (TDD), pair programming, and Agile Model Driven Development (AMDD) all have very short feedback cycles, often on the order of minutes or hours. Traditional techniques, such as reviews, inspections, and big requirements up front (BRUF) have feedback cycles on the order of weeks or months, making them riskier and expensive. (
Source: Scott Ambler)

Reducing costs from software defects requires that we shorten the feedback loop for finding and fixing those defects. As indicated in the chart included earlier, the feedback loop is quickest when using Extreme Programming methodologies and this goes for accessibility feedback as well as other quality domains. However, accessibility professionals have shown little ability to fully integrate accessibility with extreme programming practices, which means that accessibility has, in turn, remained overly expensive, time-consuming, and ineffective. Feedback must occur at all phases of product development, with high importance placed on the earliest phases. Instead, traditional accessibility testing occurs in the last stages of the project which means, in turn, that accessibility compliance becomes more costly and time consuming than it should be.

The Extreme Accessibility methodology requires a cohesive approach to testing that involves creating 6 feedback points that feed into the others.

Requirements must be defined in a way that indicate the specific business and user-interaction requirements of what is being developed. These requirements should be separated into definitions of the distinct features being developed and should be done with enough specificity that they can be used as inputs for design, development, and quality assurance feedback points. Any feature that does not make sense when documented will not make sense when used. Therefore this is the first, fastest, and easiest point in the feedback loop. Using a domain-specific language like Gherkin facilitates this.

Design must adhere to the feature requirements defined for the product. Requirement definitions are implemented here and this offers the first opportunity to verify quality of a previous feedback cycle. Applying the principles of Lean UX are especially important here. As before, issues discovered within-cycle should be handled within-cycle as well, to avoid polluting later phases with quality issues.

Development is where the bulk of accessibility issues are created, often owing to a lack of appropriate tooling. Development processes within agile environments have their own unique feedback cycle. Developers can (and should) leverage code-quality tools such as linters, syntax checkers, unit testing, and other automation to check code prior to committing it to revision control. Developers should also leverage the feature requirements specifications as an input to verifying that they’ve met the requirements for the product. Code should never be committed that doesn’t pass automated tests, because letting these defects pass into the next feedback loop will significantly increase cost and risk. It is critical, in this phase, to leverage a tool that can integrate directly into their IDEs or task runners to shorten the cycle for accessibility feedback.

Quality Assurance is a verification step for development’s automated tests and an opportunity to perform manual review. In this phase the same inputs that were generated during requirements and used during development will form the basis for acceptance tests. Automation scripts should use the same tests as those used in development. In cases where feature specifications cannot be fed into automation, QA engineers will utilize those specifications as acceptance test scripts such as with Selenium.

Continuous Integration provides an opportunity to catch regressions in code before it is deployed using the same automation scripts used by development and QA staff. This prevents critical flaws from entering production.

Review provides the ability to gather up those quality flaws that slipped through the cracks, either by virtue of sheer mistake or by knowingly taking on technical debt along the way. The review process should facilitate exploring lessons learned along each cycle in order to improve future iterations.

Though much of the above sounds similar to the “test early, test often” beliefs espoused by others in the field, the specific details – including actionable guidance, job aids and, most importantly, tooling around such process have been discussed as mere concepts. With the creation of tools like Tenon.io this no longer needs to be the case.

If you are interested in learning about the next generation in Web Accessibility Testing, give Tenon.io a try.
If you or your organization need help with accessibility consulting, strategy, or accessible web development, email me directly at karl@karlgroves.com or call me at +1 443-875-7343. Download Resume [MS Word]

2 Trackbacks