Web Accessibility Testing Tools: Who tests the DOM?

Earlier this year, I put up the Mother Effing Tool Confuser to help those who may be looking for an automated web accessibility testing tool make an educated decision. I’ve recently decided to do some of my own testing to see which tools do and don’t test the DOM. To do so, I used the Complete List of Web Accessibility Evaluation Tools from the W3C. Keep in mind, many of the tools listed are abandonware or discontinued. Also, many of them are specialty tools, such as color contrast analysis tools. I am limiting my own tool evaluations to those which do a more comprehensive test against WCAG. For each tool available, I tested the Mother Effing Tool Confuser page and, if the tool finds errors, then it isn’t testing the DOM. Simple.

At the moment, the only data I’ve gathered is for Online Services – those which allow you to submit an URL and get a report. For the most part, you can assume that browser toolbars will test the DOM, but I do plan on testing those as well and will update this blog post with that data.

Note, May 29, 2014: When I say “Test the DOM”, I’m specifically referring to assessing the DOM after rendering completely with any relevant JavaScript applied.

Online Services – tested 06-Sept 2013

Online services are those services that allow you to enter the URL for a page you wish to test and the service spits back a report. Several such tools from the W3C list no longer exist. Some services appeared down or otherwise broken and are therefore not included. For instance both the Truwex tool and the online version of FAE returned blank screens when I attempted to use them, so they aren’t listed here.

Amazingly, of those which I could use, only two actually tested the DOM: SortSite & Opquast. In other words, if you’re using a free online service to test your website’s accessibility, use SortSite or Opquast.

Tool Name Do they test the DOM?
Notes:

  1. WAVE Toolbar does test the DOM
  2. Included here mostly due to my own curiosity
aChecker No
Etre No
Accessibility Valet No
EvalAccess No
Hera No
CynthiaSays No
DaSilva No
TAW No
SortSite Yes
WAVE No 1
Opquast Yes
W3C Validator No 2
Validator.nu No 2
If you are interested in learning about the next generation in Web Accessibility Testing, sign up for the release of Tenon.io
If you or your organization need help with accessibility consulting, strategy, or accessible web development, email me directly at karl@karlgroves.com or call me at +1 443-875-7343. Download Resume [MS Word]

3 Comments

  • Posted January 1, 2014 at 11:50 am   Permalink

    I just finished reading “The Problem with Automated Accessibility Testing Tools” by Trenton Moss. From the article, with my emphasis:

    Examples of HTML-related accessibility considerations which these tools can’t check for include: [...] Making sure that the site functions without the use of JavaScript or Flash

    In that case, the tools are correctly flagging your Tool Confuser page because it does not function accessibly without the use of JavaScript. I think some of the tools that you marked “no” download the page and parse it into what the DOM looks like before any events are fired. Or they act on what the DOM looks like should googleapis.com fails to respond.

    • karlgroves
      Posted January 8, 2014 at 11:38 am   Permalink

      One thing to take note of: The article you cite was written in 2004. To put that into perspective, that’s two years before JAWS switched to a DOM engine for HTML rendering. Assistive technologies really couldn’t do much with JavaScript at the time and therefore accessibility standards at the time correctly contained an admonishment against relying on JavaScript. These days the AT landscape is vastly different and 98.6% of screen reader users browse with JavaScript enabled (Source: http://webaim.org/projects/screenreadersurvey4/#javascript).

      • Posted January 11, 2014 at 2:22 pm   Permalink

        In other words, some of the old “Until user agents including assistive technologies…” recommendations in WCAG have become fulfilled by now. Thank you for bringing me up to date on this. In that case, perhaps a better recommendation should be to include a warning “parts of this page are inaccessible without JavaScript” that is removed for these 98.6 percent of users.

Post a Comment