Web Accessibility Testing Tools: Who tests the DOM?

Earlier this year, I put up the Mother Effing Tool Confuser to help those who may be looking for an automated web accessibility testing tool make an educated decision. I’ve recently decided to do some of my own testing to see which tools do and don’t test the DOM. To do so, I used the Complete List of Web Accessibility Evaluation Tools from the W3C. I am limiting my own tool evaluations to those which do a more comprehensive test against WCAG. For each tool available, I tested the Mother Effing Tool Confuser page and, if the tool finds errors, then it isn’t testing the DOM. Simple.

At the moment, the only data I’ve gathered is for Online Services – those which allow you to submit an URL and get a report. For the most part, you can assume that browser toolbars will test the DOM, but I do plan on testing those as well and will update this blog post with that data.

Note, May 29, 2014: When I say “Test the DOM”, I’m specifically referring to assessing the DOM after rendering completely with any relevant JavaScript applied. A more accurate term might be “JavaScript-aware”

Online Services – tested 06-Sept 2013

Online services are those services that allow you to enter the URL for a page you wish to test and the service spits back a report. Several such tools from the W3C list no longer exist. Some services appeared down or otherwise broken and are therefore not included. For instance both the Truwex tool and the online version of FAE returned blank screens when I attempted to use them, so they aren’t listed here.

Amazingly, of those which I could use, only three actually tested the DOM: Tenon, SortSite & Opquast. .

Tool Name Do they test the DOM?

  1. Updated November 7, 2015
  2. Included here mostly due to my own curiosity
Tenon Yes
aChecker No
Etre No
Accessibility Valet No
EvalAccess No
Hera No
CynthiaSays No
DaSilva No
SortSite Yes
WAVE Yes 1
Opquast Yes
W3C Validator No 2
Validator.nu No 2
If you are interested in learning about the next generation in Web Accessibility Testing, give Tenon.io a try.
If you or your organization need help with accessibility consulting, strategy, or accessible web development, email me directly at karl@karlgroves.com or call me at +1 443-875-7343. Download Resume [MS Word]


  • Posted January 1, 2014 at 11:50 am   Permalink

    I just finished reading “The Problem with Automated Accessibility Testing Tools” by Trenton Moss. From the article, with my emphasis:

    Examples of HTML-related accessibility considerations which these tools can’t check for include: […] Making sure that the site functions without the use of JavaScript or Flash

    In that case, the tools are correctly flagging your Tool Confuser page because it does not function accessibly without the use of JavaScript. I think some of the tools that you marked “no” download the page and parse it into what the DOM looks like before any events are fired. Or they act on what the DOM looks like should googleapis.com fails to respond.

    • karlgroves
      Posted January 8, 2014 at 11:38 am   Permalink

      One thing to take note of: The article you cite was written in 2004. To put that into perspective, that’s two years before JAWS switched to a DOM engine for HTML rendering. Assistive technologies really couldn’t do much with JavaScript at the time and therefore accessibility standards at the time correctly contained an admonishment against relying on JavaScript. These days the AT landscape is vastly different and 98.6% of screen reader users browse with JavaScript enabled (Source: http://webaim.org/projects/screenreadersurvey4/#javascript).

      • Posted January 11, 2014 at 2:22 pm   Permalink

        In other words, some of the old “Until user agents including assistive technologies…” recommendations in WCAG have become fulfilled by now. Thank you for bringing me up to date on this. In that case, perhaps a better recommendation should be to include a warning “parts of this page are inaccessible without JavaScript” that is removed for these 98.6 percent of users.

One Trackback

  • By Competitive Analysis #1 – Tenon.io on May 16, 2015 at 9:06 pm

    […] X’s primary weakness is that it is not JavaScript-aware. Its 12 tests relating to event-handling only check for old-school HTML event handler attributes […]

Post a Comment