Element's Background Color Contrast Marked as Incomplete Instead of Violation (Cannot Determine Why)

[kopepasah]

Product

axe-core

Product Version

4.8.1

Latest Version

• [X] I have tested the issue with the latest version of the product

Issue Description

Expectation

Expected to run an e2e test with Playwright using AxeBuilder (@axe-core/playwright) and get a color-contrast issue marked as a violation.

Actual

Ran an e2e test with Playwright using AxeBuilder and color-contrast issue is marked as incomplete.

results.violations === [];

results.incomplete === [
	{
		id: 'color-contrast',
		impact: 'serious',
		tags: [
			'cat.color',
			'wcag2aa',
			'wcag143',
			'TTv5',
			'TT13.c',
			'EN-301-549',
			'EN-9.1.4.3',
			'ACT',
		],
		description:
			'Ensures the contrast between foreground and background colors meets WCAG 2 AA minimum contrast ratio thresholds',
		help: 'Elements must meet minimum color contrast ratio thresholds',
		helpUrl:
			'https://dequeuniversity.com/rules/axe/4.8/color-contrast?application=playwright',
		nodes: [
			{
				any: [
					{
						id: 'color-contrast',
						data: {
							contrastRatio: 0,
							fontSize: '10.5pt (14px)',
							fontWeight: 'normal',
							messageKey: 'bgOverlap',
							expectedContrastRatio: '4.5:1',
						},
						relatedNodes: [],
						impact: 'serious',
						message:
							"Element's background color could not be determined because it is overlapped by another element",
					},
				],
				all: [],
				none: [],
				impact: 'serious',
				html: '<div>Many or all companies we feature compensate us. Compensation and editorial <br class="d-none d-lg-block"> research influence how products appear on a page.</div>',
				target: [ '.disclaimer > div' ],
				failureSummary:
					'Fix any of the following:\n' +
					"  Element's background color could not be determined because it is overlapped by another element",
			},
		],
	},
];

How to Reproduce

Run the following test using Playwright with @axe-core/playwright:

import { test, expect } from '@playwright/test';
import AxeBuilder from '@axe-core/playwright';

test( 'test disclaimer has correct color contrast', async ( { page } ) => {
	await page.goto( 'https://lendedu.com/blog/private-student-loans/' );

	await page.locator( '.disclaimer' ).waitFor();

	const results = await new AxeBuilder( { page } )
		.include( '.disclaimer' )
		.analyze();

	expect( results.violations ).toEqual( [] );
} );

Additional context

Initially, I worked on this locally and made the local div.disclaimer a contrast violation. However, the results still mark it as incomplete. I am not able to determine why this is marked as incomplete, as there is no overlapping element (as described in the incomplete object).

In addition, I tried targeting the div directly (i.e. .include( '.disclaimer > div' )), and also tried removing the childdiv and br from div.disclaimer, but all result in the same incomplete notice.

[kopepasah]

Also, I considered opening this issue in https://github.com/dequelabs/axe-core-npm, but decided this repo was correct, as it does not seem to be related to Playwright.

Let me know if it needs to be moved.

[straker]

Thanks for the issue. I can't say for certain why it's being marked as incomplete. Would you be able to share the website this happened on? If not, maybe a codepen that demonstrates the issue?

[kopepasah]

Hello @straker! Thanks for the prompt response.

I am not sure how to run this Playwright test within Codepen, but created one using the "How to Reproduce" code above.

The URL which Playwright is testing from: https://lendedu.com/blog/private-student-loans/

If you have Playwright (@playwright/test) and (@axe-core/playwright) installed locally, and run the test listed above, you can reproduce the issue, listing the color-contrast test as incomplete.

Please let me know if you need more information.

[straker]

Thanks for the website, I can see what's going on. The issue is in our code and is caused by the body having a z-index: -1.

Mostly for my notes: When we try to calculate the element stack for the disclaimer div, our code comes back with the html element at the top of stack rather than the bottom due to the z-index of the body. Reproducible small test case:

<html>
<body style="z-index: -1; position: relative;">
  <div>Text</div>
</body>
</html>

[kopepasah]

@straker thanks so much for the additional information and help! 🙇🏻