{"id":210,"date":"2019-06-04T13:17:01","date_gmt":"2019-06-04T17:17:01","guid":{"rendered":"https:\/\/pressbooks.library.ryerson.ca\/iwacc\/?post_type=chapter&#038;p=210"},"modified":"2022-12-16T10:26:38","modified_gmt":"2022-12-16T15:26:38","slug":"activity-4-understanding-the-limitations-of-automated-accessibility-checkers","status":"publish","type":"chapter","link":"https:\/\/pressbooks.library.torontomu.ca\/iwacc\/chapter\/activity-4-understanding-the-limitations-of-automated-accessibility-checkers\/","title":{"raw":"Activity 4: Understanding the Limitations of Automated Accessibility Checkers","rendered":"Activity 4: Understanding the Limitations of Automated Accessibility Checkers"},"content":{"raw":"<div class=\"colorBox activity\">\r\n\r\n<img src=\"https:\/\/pressbooks.library.ryerson.ca\/iwacc\/wp-content\/uploads\/sites\/82\/2019\/06\/activity.png\" alt=\"\" class=\"size-full wp-image-200 alignright\" width=\"76\" height=\"80\" \/>There are a variety of tools available on the Internet, as well as plugins or add-ons for web browsers, that can be used to test the accessibility of web content. But it is important to know that these tools differ in the accuracy and coverage of what they test.\r\n\r\nWhile automated accessibility checkers are a great way to get a quick review of a website\u2019s accessibility, they cannot be relied upon to identify all potential barriers or even to identify them accurately. Some barriers, particularly those that involve meaning in one way or another, can\u2019t be measured with automated checkers (at least, not with the current state of the technology). Checkers are also not able to determine whether some types of inaccessible content have accessible alternatives.\r\n<h3>Activity<\/h3>\r\nIn this activity, you will look at two popular automated accessibility testing tools, AChecker and Wave, plus another one of your choosing. You will describe what they are identifying and not identifying as barriers. Some tools are quite clear about what they test and list the checks they run. Other tools hide away the checks from the end user, making it difficult to know exactly what is being tested. Some have hundreds of checks they run. Others have just a handful. Some checkers are customizable to the needs of each user; others have little or no customization. The bottom line is accessibility checkers are not created equal.\r\n<div style=\"margin: 1em 0;padding: 1em;border: 1px solid #ddd;border-left: 10px solid #f66\"><strong>Key Point: <\/strong> Do not confuse checks with guidelines or success criteria. Checks are typically more granular. Take SC 1.1.1. Checkers can check for the presence of the string \u201calt\u201d when it finds an HTML image element. Another check looks for a value for alt, and another the length of that value. Another check looks for the presence of a title attribute, or perhaps an aria-describedby attribute. All of these checks, and others, are related to this particular success criteria. A success criteria may have a dozen or more checks associated with it.<\/div>\r\n<h4>Accessibility Checkers<\/h4>\r\n<ul>\r\n \t<li><a href=\"https:\/\/achecker.achecks.ca\/checker\/index.php\">AChecker<\/a><\/li>\r\n \t<li><a href=\"https:\/\/wave.webaim.org\/\">Wave<\/a><\/li>\r\n \t<li>Choose one other from those listed in \u201cOther Accessibility Checkers\u201d (below)<\/li>\r\n<\/ul>\r\n<h4>Test Sites (Homepage only)<\/h4>\r\n<ul>\r\n \t<li><a href=\"https:\/\/de.torontomu.ca\/wa\/showcase\/\" target=\"_blank\" rel=\"noopener\">Accessibility Auditing Showcase<\/a><\/li>\r\n \t<li><a href=\"https:\/\/de.torontomu.ca\/wa\/lulu\/\" target=\"_blank\" rel=\"noopener\">Lulu\u2019s Lollipops<\/a><\/li>\r\n<\/ul>\r\n<h4>Other Accessibility Checkers<\/h4>\r\n<ul>\r\n \t<li><a href=\"https:\/\/dynomapper.com\/blog\/27-accessibility-testing\/246-top-25-awesome-accessibility-testing-tools-for-websites\">Top 25 Awesome Accessibility Testing Tools for Websites<\/a><a href=\"https:\/\/dynomapper.com\/blog\/27-accessibility-testing\/246-top-25-awesome-accessibility-testing-tools-for-websites\"><\/a><\/li>\r\n \t<li><a href=\"https:\/\/alphagov.github.io\/accessibility-tool-audit\/index.html\">How do automated accessibility checkers compare?<\/a><\/li>\r\n<\/ul>\r\n<h3>Requirements<\/h3>\r\nUsing the homepage from the two test sites listed above (i.e., Showcase and Lulu\u2019s), compare the reports generated by AChecker, Wave, and an accessibility checker of your choice (being sure to name it).\r\n\r\nAnswer the following questions:\r\n<ol>\r\n \t<li>How many known accessibility issues does each checker identify on each of the test sites\u2019 homepages?<\/li>\r\n \t<li>Comparing each report, what did each checker miss that one of the others may have caught? Provide specific examples.<\/li>\r\n \t<li>How many manual checks did each checker suggest? (Manual checks would be checks a human needs to make.)<\/li>\r\n \t<li>Were there any false positives? (Examples of false positives include: identifying barriers that are not barriers or identifying barriers that have an accessible alternative available.)<\/li>\r\n \t<li>Does the checker list the checks it runs? (This may take a little research or digging around the settings or options of the application. Also, see the note above that describes what a check is, as opposed to a guideline or success criteria.)<\/li>\r\n \t<li>Based on your experience here with the three checkers, what are your overall thoughts on their accuracy and coverage?<\/li>\r\n<\/ol>\r\n<div style=\"margin: 1em 0;padding: 1em;border: 1px solid #ddd;border-left: 10px solid #f66\"><strong>Key Point: <\/strong> Do not assume the reports generated by the checkers are accurate when comparing them in <strong>Question 2<\/strong> above. Confirm for yourself that the items you selected for this question are being accurately reported.<em><strong> Warning:<\/strong><\/em> This might involve looking at HTML.<\/div>\r\n<\/div>","rendered":"<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_80 counter-hierarchy ez-toc-counter ez-toc-grey ez-toc-container-direction\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Contents<\/p>\n<label for=\"ez-toc-cssicon-toggle-item-69d0d2fedf9f7\" class=\"ez-toc-cssicon-toggle-label\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #999;color:#999\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #999;color:#999\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/label><input type=\"checkbox\"  id=\"ez-toc-cssicon-toggle-item-69d0d2fedf9f7\"  aria-label=\"Toggle\" \/><nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/pressbooks.library.torontomu.ca\/iwacc\/chapter\/activity-4-understanding-the-limitations-of-automated-accessibility-checkers\/#Activity\" >Activity<\/a><ul class='ez-toc-list-level-4' ><li class='ez-toc-heading-level-4'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/pressbooks.library.torontomu.ca\/iwacc\/chapter\/activity-4-understanding-the-limitations-of-automated-accessibility-checkers\/#Accessibility_Checkers\" >Accessibility Checkers<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-4'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/pressbooks.library.torontomu.ca\/iwacc\/chapter\/activity-4-understanding-the-limitations-of-automated-accessibility-checkers\/#Test_Sites_Homepage_only\" >Test Sites (Homepage only)<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-4'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/pressbooks.library.torontomu.ca\/iwacc\/chapter\/activity-4-understanding-the-limitations-of-automated-accessibility-checkers\/#Other_Accessibility_Checkers\" >Other Accessibility Checkers<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/pressbooks.library.torontomu.ca\/iwacc\/chapter\/activity-4-understanding-the-limitations-of-automated-accessibility-checkers\/#Requirements\" >Requirements<\/a><\/li><\/ul><\/nav><\/div>\n<div class=\"colorBox activity\">\n<p><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/pressbooks.library.ryerson.ca\/iwacc\/wp-content\/uploads\/sites\/82\/2019\/06\/activity.png\" alt=\"\" class=\"size-full wp-image-200 alignright\" width=\"76\" height=\"80\" \/>There are a variety of tools available on the Internet, as well as plugins or add-ons for web browsers, that can be used to test the accessibility of web content. But it is important to know that these tools differ in the accuracy and coverage of what they test.<\/p>\n<p>While automated accessibility checkers are a great way to get a quick review of a website\u2019s accessibility, they cannot be relied upon to identify all potential barriers or even to identify them accurately. Some barriers, particularly those that involve meaning in one way or another, can\u2019t be measured with automated checkers (at least, not with the current state of the technology). Checkers are also not able to determine whether some types of inaccessible content have accessible alternatives.<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Activity\"><\/span>Activity<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>In this activity, you will look at two popular automated accessibility testing tools, AChecker and Wave, plus another one of your choosing. You will describe what they are identifying and not identifying as barriers. Some tools are quite clear about what they test and list the checks they run. Other tools hide away the checks from the end user, making it difficult to know exactly what is being tested. Some have hundreds of checks they run. Others have just a handful. Some checkers are customizable to the needs of each user; others have little or no customization. The bottom line is accessibility checkers are not created equal.<\/p>\n<div style=\"margin: 1em 0;padding: 1em;border: 1px solid #ddd;border-left: 10px solid #f66\"><strong>Key Point: <\/strong> Do not confuse checks with guidelines or success criteria. Checks are typically more granular. Take SC 1.1.1. Checkers can check for the presence of the string \u201calt\u201d when it finds an HTML image element. Another check looks for a value for alt, and another the length of that value. Another check looks for the presence of a title attribute, or perhaps an aria-describedby attribute. All of these checks, and others, are related to this particular success criteria. A success criteria may have a dozen or more checks associated with it.<\/div>\n<h4><span class=\"ez-toc-section\" id=\"Accessibility_Checkers\"><\/span>Accessibility Checkers<span class=\"ez-toc-section-end\"><\/span><\/h4>\n<ul>\n<li><a href=\"https:\/\/achecker.achecks.ca\/checker\/index.php\">AChecker<\/a><\/li>\n<li><a href=\"https:\/\/wave.webaim.org\/\">Wave<\/a><\/li>\n<li>Choose one other from those listed in \u201cOther Accessibility Checkers\u201d (below)<\/li>\n<\/ul>\n<h4><span class=\"ez-toc-section\" id=\"Test_Sites_Homepage_only\"><\/span>Test Sites (Homepage only)<span class=\"ez-toc-section-end\"><\/span><\/h4>\n<ul>\n<li><a href=\"https:\/\/de.torontomu.ca\/wa\/showcase\/\" target=\"_blank\" rel=\"noopener\">Accessibility Auditing Showcase<\/a><\/li>\n<li><a href=\"https:\/\/de.torontomu.ca\/wa\/lulu\/\" target=\"_blank\" rel=\"noopener\">Lulu\u2019s Lollipops<\/a><\/li>\n<\/ul>\n<h4><span class=\"ez-toc-section\" id=\"Other_Accessibility_Checkers\"><\/span>Other Accessibility Checkers<span class=\"ez-toc-section-end\"><\/span><\/h4>\n<ul>\n<li><a href=\"https:\/\/dynomapper.com\/blog\/27-accessibility-testing\/246-top-25-awesome-accessibility-testing-tools-for-websites\">Top 25 Awesome Accessibility Testing Tools for Websites<\/a><a href=\"https:\/\/dynomapper.com\/blog\/27-accessibility-testing\/246-top-25-awesome-accessibility-testing-tools-for-websites\"><\/a><\/li>\n<li><a href=\"https:\/\/alphagov.github.io\/accessibility-tool-audit\/index.html\">How do automated accessibility checkers compare?<\/a><\/li>\n<\/ul>\n<h3><span class=\"ez-toc-section\" id=\"Requirements\"><\/span>Requirements<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>Using the homepage from the two test sites listed above (i.e., Showcase and Lulu\u2019s), compare the reports generated by AChecker, Wave, and an accessibility checker of your choice (being sure to name it).<\/p>\n<p>Answer the following questions:<\/p>\n<ol>\n<li>How many known accessibility issues does each checker identify on each of the test sites\u2019 homepages?<\/li>\n<li>Comparing each report, what did each checker miss that one of the others may have caught? Provide specific examples.<\/li>\n<li>How many manual checks did each checker suggest? (Manual checks would be checks a human needs to make.)<\/li>\n<li>Were there any false positives? (Examples of false positives include: identifying barriers that are not barriers or identifying barriers that have an accessible alternative available.)<\/li>\n<li>Does the checker list the checks it runs? (This may take a little research or digging around the settings or options of the application. Also, see the note above that describes what a check is, as opposed to a guideline or success criteria.)<\/li>\n<li>Based on your experience here with the three checkers, what are your overall thoughts on their accuracy and coverage?<\/li>\n<\/ol>\n<div style=\"margin: 1em 0;padding: 1em;border: 1px solid #ddd;border-left: 10px solid #f66\"><strong>Key Point: <\/strong> Do not assume the reports generated by the checkers are accurate when comparing them in <strong>Question 2<\/strong> above. Confirm for yourself that the items you selected for this question are being accurately reported.<em><strong> Warning:<\/strong><\/em> This might involve looking at HTML.<\/div>\n<\/div>\n","protected":false},"author":100,"menu_order":12,"template":"","meta":{"pb_show_title":"on","pb_short_title":"","pb_subtitle":"","pb_authors":[],"pb_section_license":""},"chapter-type":[],"contributor":[],"license":[],"class_list":["post-210","chapter","type-chapter","status-publish","hentry"],"part":28,"_links":{"self":[{"href":"https:\/\/pressbooks.library.torontomu.ca\/iwacc\/wp-json\/pressbooks\/v2\/chapters\/210","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/pressbooks.library.torontomu.ca\/iwacc\/wp-json\/pressbooks\/v2\/chapters"}],"about":[{"href":"https:\/\/pressbooks.library.torontomu.ca\/iwacc\/wp-json\/wp\/v2\/types\/chapter"}],"author":[{"embeddable":true,"href":"https:\/\/pressbooks.library.torontomu.ca\/iwacc\/wp-json\/wp\/v2\/users\/100"}],"version-history":[{"count":15,"href":"https:\/\/pressbooks.library.torontomu.ca\/iwacc\/wp-json\/pressbooks\/v2\/chapters\/210\/revisions"}],"predecessor-version":[{"id":1370,"href":"https:\/\/pressbooks.library.torontomu.ca\/iwacc\/wp-json\/pressbooks\/v2\/chapters\/210\/revisions\/1370"}],"part":[{"href":"https:\/\/pressbooks.library.torontomu.ca\/iwacc\/wp-json\/pressbooks\/v2\/parts\/28"}],"metadata":[{"href":"https:\/\/pressbooks.library.torontomu.ca\/iwacc\/wp-json\/pressbooks\/v2\/chapters\/210\/metadata\/"}],"wp:attachment":[{"href":"https:\/\/pressbooks.library.torontomu.ca\/iwacc\/wp-json\/wp\/v2\/media?parent=210"}],"wp:term":[{"taxonomy":"chapter-type","embeddable":true,"href":"https:\/\/pressbooks.library.torontomu.ca\/iwacc\/wp-json\/pressbooks\/v2\/chapter-type?post=210"},{"taxonomy":"contributor","embeddable":true,"href":"https:\/\/pressbooks.library.torontomu.ca\/iwacc\/wp-json\/wp\/v2\/contributor?post=210"},{"taxonomy":"license","embeddable":true,"href":"https:\/\/pressbooks.library.torontomu.ca\/iwacc\/wp-json\/wp\/v2\/license?post=210"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}