The following data comes from automated accessibility testing of the Alexa Top 100 US websites (minus the pr0n, search engines, social networks and sites primarily driven from user content) using AQUA and lists their performance from worst to best (based on density of errors per page). This information comes with the important caveat that it is limited to automated testing. That being said, my own experience has been that poor performance in automated accessibility testing is strongly correlated with poor performance in manual accessibility testing as well.
| Site Name |
Errors | Warnings | Pages Tested | Avg Per Page (Errors + Warnings) |
Avg. Per Page (Errors Only) |
|---|---|---|---|---|---|
| LA Times | 75 | 46 | 1 | 121.00 | 75.00 |
| Wal Mart | 23691 | 9404 | 500 | 66.19 | 47.38 |
| IMDB | 22746 | 35552 | 496 | 117.54 | 45.86 |
| CNN | 22255 | 18354 | 498 | 81.54 | 44.69 |
| Answers | 21663 | 2477 | 500 | 48.28 | 43.33 |
| Roadrunner | 183772 | 131066 | 4979 | 63.23 | 36.91 |
| Huffington Post | 17652 | 2697 | 499 | 40.78 | 35.37 |
| Best Buy | 383 | 1886 | 12 | 189.08 | 31.92 |
| Fox News | 11456 | 5478 | 361 | 46.91 | 31.73 |
| Weather Channel | 13868 | 13035 | 491 | 54.79 | 28.24 |
| Washington Post | 12850 | 19892 | 464 | 70.56 | 27.69 |
| NFL | 12104 | 27059 | 498 | 78.64 | 24.31 |
| CNET | 11863 | 34408 | 499 | 92.73 | 23.77 |
| eHow | 3987 | 1464 | 199 | 27.39 | 20.04 |
| Wall St. Journal | 201 | 243 | 16 | 27.75 | 12.56 |
| New York Times | 6085 | 6458 | 496 | 25.29 | 12.27 |
| Microsoft | 4824 | 15804 | 431 | 47.86 | 11.19 |
| Apple | 4690 | 6705 | 500 | 22.79 | 9.38 |
| Chase | 2908 | 848 | 361 | 10.40 | 8.06 |
| BBC | 3823 | 10364 | 483 | 29.37 | 7.92 |
| PayPal | 2992 | 4822 | 498 | 15.69 | 6.01 |
| AOL | 60 | 56 | 10 | 11.60 | 6.00 |
| About.com | 833 | 401 | 142 | 8.69 | 5.87 |
| Netflix | 2544 | 1814 | 474 | 9.19 | 5.37 |
| Verizon Wireless | 1447 | 940 | 275 | 8.68 | 5.26 |
| Comcast | 1803 | 745 | 368 | 6.92 | 4.90 |
| USPS | 1158 | 3289 | 338 | 13.16 | 3.43 |
| Wells Fargo | 1176 | 3831 | 479 | 10.45 | 2.46 |
| UPS | 2 | 272 | 1 | 274.00 | 2.00 |
| Bank of America | 489 | 6316 | 314 | 21.67 | 1.56 |
Some Definitions
- Error
- In this context, an “error” is something that was found using a test which us clearly ‘pass/fail’ in nature. Things such as missing alt attributes for images fall into this category.
- Warning
- In this context, a “warning” is something that was found using a test which does not have a clear ‘pass/fail’ criteria but would instead require human inspection to verify.
If you or your organization need help with accessibility consulting, strategy, or accessible web development, contact Simply Accessible or email me directly at karl@simplyaccessible.com or call me at +1 443-875-7343
If you enjoyed this article, please consider sharing it!
7 Responses to Alexa Top 100 Accessibility
Leave a Reply
You must be logged in to post a comment.
The danger here is that without context, some could look at these stats and make some sweeping judgments about how accessible certain sites are over others. The fact that there is such a variability in the number of pages tested, coupled with the fact that we have no idea about whether pages tested were plain HTML or built using rich internet technologies, or both (which could be within a site or between sites) makes me discomforted with such a ranking without much more information.
Jennison, I would concede that in a purely scientific setting it would be important to supply additional details with this data including exactly when testing happened, what tool was used, what tests were performed, exactly which pages were tested, etc. In practice, however, I’ve yet to experience a website who performed poorly in automated testing but performed well in manual or use case testing. The converse, admittedly, can often be true. In the future I plan on doing something that calculates error density-per-page using the document’s size.
I humbly submit a few questions.
What tool was used to test these pages?
How were the pages selected?
Why such variability in the number of pages tested?
What were these pages tested against (e.g., WCAG 2.0 AA, Section 508)?
I tested these sites (as well as most US Federal Government websites) using AQUA, a tool I’ve been developing for a long time. Over the last year or so, I’ve been working on revamping the way automated testing works – specifically when it comes to speed. The tests weren’t meant to be a test of these sites but rather a test of AQUA’s ability to spider & test rapidly and accurately. I chose these sites because they would be large and complex which afforded me the opportunity to really throttle the spidering capability. I set it to test up to 500 pages per site. Anything less than 500 means that it reached as many pages as it could before experiencing one of any number of items which would cause it to stop (the list of things which would cause it to stop is too numerous and boring to go into here). There’s a weird anomaly with Roadrunner that made it go far beyond 500 on that site. These were tests of WCAG 2.0 AA.
Hey, finally got to register to this joint.
Just meant to say your last sentence really made me smile: “…poor performance in automated testing is strongly correlated with poor performance in manual testing as well.”
That’s a hell of an understatement. “Strongly” doesn’t even begin to describe it. Even more so – I’d say that on websites where automated testing fails, it’s actually a waste of everybody’s time and money to bother going even further.
I really feel that until we’ve reached the point where automated testing still fails, there’s no real point in evaluating for anything else. The basics need to be covered first.
[...] Groves recently published automated web accessibility test data for many of the Alexa Top 100 web sites. The results paint a rather stark picture of web [...]
[...] have recently come across an article by KARLGROVES, Alexa Top 100 Accessibility and Alexa 100 Accessibility Errors by Jared Smith as well. After reading it I thought of putting [...]