axe-core icon indicating copy to clipboard operation
axe-core copied to clipboard
verified publisher icon dequelabs

Color Contrast rules for WCAG2.0 And WCGA2.1 giving incorrect result.

Open praveshpr opened this issue 1 year ago • 5 comments

I am using axe-core library in node js with playwright.

Details:-

Install Node.js if you haven't already.

Install Playwright: npm install playwright

Install @axe-core/playwright: npm install @axe-core/playwright

v4.10

I have below questions. 1.What default rule Axe-core uses to scan the website for accessibility ?

Issue:- 2.When I am using axe core to scan web page I noticed that It is giving issue for Color contrast. It is showing expected color contrast should be 3, but it found 1.9.2 and apart from this I got other issues as well. I have concern about this[I read some where in article that Default scan tag used by axe core is WCGA2.0, is that correct? if yes then is that color contrast rules are different form WCAG2.0 & WCAG2.1? if yes then what is the different?] And same when I scan the page using tags "wcag22aa","wcag21aa",then my page does not show any accessibility issues.

Now I wanted to understand this, what happens why it gave error with default scan and why not with wcag22aa","wcag21aa" tags? What does it mean?

3.How accurate the scan results is with axe-core? in terms of %.

4.Can we take it as 100% accessibility free it it passes certain scan rule?

praveshpr avatar Sep 14 '24 23:09 praveshpr

Thanks for the questions.

  1. Axe-core runs all WCAG 2.0 and 2.1, A and AA rules except those with tags experimental and deprecated. WCAG 2.2 and WCAG AAA rules are not run by default.
  2. Would it be possible to see the page where the error is happening? Without that it will be hard to tell why it's giving color contrast issues. As for the reason it doesn't throw issues for wcag22aa, wcag21aa tags, that is because color contrast is a wcag2aa rule. The way the tags work in axe-core is you have to pass all the tags you want enabled. For example, wcag22aa does not enable wcag21aa nor wcag2aa. You'll have to pass each of those for those rules to run.
  3. Axe-core aims for a 0 false positive rate, so when axe-core reports a violation it typically means it's a real problem.
  4. Automated testing cannot guarantee that a web page is 100% accessible. Automated test can only capture ~60% of all accessibility problems. The remaining problems would have to be found by manual investigation and testing the page. Furthermore, automated tests are not always able to 100% test any given WCAG rule, so even if the axe-core result passes the rule, it doesn't mean there are no problems.

straker avatar Sep 16 '24 15:09 straker

Thanks team, Here are the details. When I am scanning page without any tags, it gives me below issue.

Object { + "description": "Ensure links are distinguished from surrounding text in a way that does not rely on color", + "help": "Links must be distinguishable without relying on color", + "helpUrl": " https://dequeuniversity.com/rules/axe/4.10/link-in-text-block?application=playwright ", + "id": "link-in-text-block", + "impact": "serious", + "nodes": Array [ + Object { + "all": Array [], + "any": Array [ + Object { + "data": Object { + "contrastRatio": 1.92, + "messageKey": "fgContrast", + "nodeColor": "#0074bc", + "parentColor": "#3a474e", + "requiredContrastRatio": 3, + }, + "id": "link-in-text-block", + "impact": "serious", + "message": "The link has insufficient color contrast of 1.92:1 with the surrounding text. (Minimum contrast is 3:1, link text: #0074bc, surrounding text: #3a474e)", + "relatedNodes": Array [ + Object { + "html": "<div class="alert_component_core-alert-container__CY3ZE">", + "target": Array [ + ".alert_component_core-alert-container__CY3ZE", + ], + }, + ], + }, + Object { + "data": null, + "id": "link-in-text-block-style", + "impact": "serious", + "message": "The link has no styling (such as underline) to distinguish it from the surrounding text", + "relatedNodes": Array [ + Object { + "html": "<div class="alert_component_core-alert-container__CY3ZE">", + "target": Array [ + ".alert_component_core-alert-container__CY3ZE", + ], + }, + ], + }, + ], + "failureSummary": "Fix any of the following: + The link has insufficient color contrast of 1.92:1 with the surrounding text. (Minimum contrast is 3:1, link text: #0074bc, surrounding text: #3a474e) + The link has no styling (such as underline) to distinguish it from the surrounding text", + "html": "<a href=" https://www.woolworthsgroup.com.au/au/en/privacy/policy-documents/privacy-policy.html" class="alert_component_alert-link-informational__4mi6S undefined trailing" target="_blank" aria-label="Link to Privacy Policy page. Opens in new Window.">", + "impact": "serious", + "none": Array [], + "target": Array [ + ".alert_component_alert-link-informational__4mi6S", + ], + }, + ], + "tags": Array [ + "cat.color", + "wcag2a", + "wcag141", + "TTv5", + "TT13.a", + "EN-301-549", + "EN-9.1.4.1", + ],

But when I scan the same page with tags "wcag22aa","wcag21aa" then I don't get any issue. Now I wanted to know why they are behaving differently?

What rule or tag Playwright axe-core uses by default to scan the the page? And why the above colour contrast issue is not caught when I am scanning application with tags wcag22aa","wcag21aa

Regards, Pravesh P.

On Tue, Sep 17, 2024 at 1:25 AM Steven Lambert @.***> wrote:

Thanks for the questions.

  1. Axe-core runs all WCAG 2.0 and 2.1, A and AA rules https://github.com/dequelabs/axe-core/blob/develop/doc/rule-descriptions.md except those with tags experimental and deprecated. WCAG 2.2 and WCAG AAA rules are not run by default.
  2. Would it be possible to see the page where the error is happening? Without that it will be hard to tell why it's giving color contrast issues. As for the reason it doesn't throw issues for wcag22aa, wcag21aa tags, that is because color contrast is a wcag2aa rule. The way the tags work in axe-core is you have to pass all the tags you want enabled. For example, wcag22aa does not enable wcag21aa nor wcag2aa. You'll have to pass each of those for those rules to run.
  3. Axe-core aims for a 0 false positive rate, so when axe-core reports a violation it typically means it's a real problem.
  4. Automated testing cannot guarantee that a web page is 100% accessible. Automated test can only capture ~60% of all accessibility problems. The remaining problems would have to be found by manual investigation and testing the page. Furthermore, automated tests are not always able to 100% test any given WCAG rule, so even if the axe-core result passes the rule, it doesn't mean there are no problems.

— Reply to this email directly, view it on GitHub https://github.com/dequelabs/axe-core/issues/4579#issuecomment-2353234572, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADCIK6PUPORMHWGZDQV5LZDZW3Z53AVCNFSM6AAAAABOHH6XJGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNJTGIZTINJXGI . You are receiving this because you authored the thread.Message ID: @.***>

-- Regards, Pravesh prajapati. Mob:-9702600170

praveshpr avatar Sep 19 '24 22:09 praveshpr

The way the tags work in axe-core is you have to pass all the tags you want enabled. For example, wcag22aa does not enable wcag21aa nor wcag2aa. You'll have to pass each of those for those rules to run.

color-contrast is a wcag2a rule, and the issues above are from the link-in-text-bock rule, which is also a wcag2a rule. When you pass tags to be run in axe-core, you have to pass all tags you wanted run. Since you did not pass wcag2a as a tag it did not run those rules.

So if you wanted to run all WCAG A and AA rules, you'd need to pass the following tags (we don't have any wcag22a rules so it's not passed):

wcag2a, wcag2aa, wcag21a, wcag21aa, wcag22aa

straker avatar Sep 20 '24 14:09 straker

Hi Team, Thanks for reply, Still unclear for me.

Can you explain to me if we don't pass any tag, then on what rules It scans the page and gives output? There might be some default rule or tag, based on that It will scan pages when no tags is passed?

If I understand correctly do you mean to say that, axe-core uses only wcag2a tag to scan pages , if no tags were passed?

And the line you mention

color-contrast is a wcag2a rule, and the issues above are from the link-in-text-bock rule, which is also a wcag2a rule. When you pass tags to be run in axe-core, you have to pass all tags you wanted run. Since you did not pass wcag2a as a tag it did not run those rules.

What does the above statement mean?

you mean to say color-contrast is a wcag2a rule, and the issues above are from the link-in-text-bock rule ? is not part of wcag22a & wcag21aa

When I saw on documentation color-contrast is a wcag2a rule, and the link-in-text-bock rule is also part of wcag22a & wcag21aa, then why it was not detected when I scan with these 2 tags? wcag22a & wcag21aa?

Can you call me to discuss this?

Mob:- +61468345103

Regards, Pravesh P.

On Sat, Sep 21, 2024 at 12:35 AM Steven Lambert @.***> wrote:

The way the tags work in axe-core is you have to pass all the tags you want enabled. For example, wcag22aa does not enable wcag21aa nor wcag2aa. You'll have to pass each of those for those rules to run.

color-contrast is a wcag2a rule, and the issues above are from the link-in-text-bock rule, which is also a wcag2a rule. When you pass tags to be run in axe-core, you have to pass all tags you wanted run. Since you did not pass wcag2a as a tag it did not run those rules.

So if you wanted to run all WCAG A and AA rules, you'd need to pass the following tags (we don't have any wcag22a rules so it's not passed):

wcag2a, wcag2aa, wcag21a, wcag21aa, wcag22aa

— Reply to this email directly, view it on GitHub https://github.com/dequelabs/axe-core/issues/4579#issuecomment-2363883965, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADCIK6LR5B7FDIP4XFLIUYTZXQXEPAVCNFSM6AAAAABOHH6XJGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNRTHA4DGOJWGU . You are receiving this because you authored the thread.Message ID: @.***>

-- Regards, Pravesh prajapati. Mob:-9702600170

praveshpr avatar Sep 21 '24 02:09 praveshpr

When axe-core runs, it by default will run all axe-core rules that have the tag wcag2a, wcag2aa, wcag21a, or wcag21aa. Each axe-core rule is tagged with only a single WCAG version which is when the WCAG rule was first introduced and not necessarily in which of the versions the rule is part of.

So for example, the button-name rule, which is based on WCAG 4.1.2, was first introduced in the WCAG 2.0 specification. So even though the rule applies to the WCAG 2.0 (A), WCAG 2.1 (A), and WCAG 2.2 (A) standards, the axe-core button-name rule itself is tagged only as wcag2a (since it was first introduced in WCAG 2.0).

What that means is that in order to run the button-name rule, the wcag2a tag must be passed to axe-core when you pass it tags. So if you only pass the wcag21a tag, no wcag2a axe-core rules will be run.

To see which axe-core rules use what tags, you can look at the "Tag" column for the rule in our rule descriptions documentation.

Going back to the output of the issues you posted, they look like they are from the link-in-text-block rule (the help url all point to https://dequeuniversity.com/rules/axe/4.10/link-in-text-block?application=playwright). The link-in-text-block rule checks that text links can be distinguished from the surrounding text. The rule checks the color contrast of the link to the surrounding text to determine if the link uses a contrast that is greater than 3:1. If the links do not pass the color contrast check, and the links do not use any other visual styles to distinguish them, we report violations for the link that the contrast is not sufficient.

The axe-core link-in-text-block rule is a different rule than the axe-core color-contrast rule. link-in-text-block is tagged with wcag2a. This is why when you run the page using only wcag22aa and wcag21aa tags, the rule does not run and you don't see any violations dealing with color contrast (i.e. link contrast from the link-in-text-block rule).

straker avatar Sep 23 '24 14:09 straker