accessibility-insights-windows
accessibility-insights-windows copied to clipboard
microsoft
AI for Windows is unable to detect text color for short text strings
Please check whether the bug has already been filed.
Describe the bug I noticed that the auto-detect color contrast feature in the latest Accessibility Insights win32 app is unable to detect colors in our app UI or content in the web container.
To Reproduce Steps to reproduce the behavior:
- Open Microsoft Edge (Anaheim)
- Open AI for Windows
- Attempt to detect color contrast ratio for a button on the application toolbar
Expected behavior AI for windows accurate determines text color and background fill color
Actual behavior AI for windows says it is unable to detect colors
thanks for the bug @roryfa! From looking at this use case, testing icons is not an available functionality for the auto-detect feature. From our documentation on Color Contrast: "Because auto detect uses a heuristic intended specifically for text, the contrast ratio of UI components and graphical elements must be tested manually"
The team requires additional author feedback; please review their replies and update this issue accordingly. Thank you for contributing to Accessibility Insights!
I spoke with @roryfa, and while antialiasing is happening here, it's not the main issue. The main issue is that the color contrast algorithm is designed to work on a series of characters, where it detects transitions between the foreground and background color as it runs across a line. It needs a minimum number of transitions to trigger a match, so a string like "OK" would be less likely to succeed than a string like "OK, I understand".
Here's a zoomed image of the bitmap being used for this issue:
About half of the scanlines of this bitmap have only 1 color transition, and those with more than 1 just don't provide enough information for the scanline algorithm to successfully detect the colors. I'll convert this to a feature request and put it into the backlog.
This issue requires additional investigation by the Accessibility Insights team. When the issue is ready to be triaged again, we will update the issue with the investigation result and add "status: ready for triage". Thank you for contributing to Accessibility Insights!
This issue has been marked as ready for team triage; we will triage it in our weekly review and update the issue. Thank you for contributing to Accessibility Insights!
I have a branch where I've rewritten the color contrast algorithm. We'll see if we can get this ready to ship
This issue has been marked as ready for team triage; we will triage it in our weekly review and update the issue. Thank you for contributing to Accessibility Insights!
This isn't really in an active state, so we're restarting the triage process on this
+1 for this getting into the production version!
I just filed a question on this and learned of this bug today. We're running into issues where the new algo might be helpful.
Currently, the tool is providing strange color selections that are actually colors from the antialiasing applied by ClearType. So, even though our code may say the text is black over white, the tool automatically detects a purple color over white. We're having a rough time making things accessible since the tool is giving odd results.
@salux, you can also check color contrast by using the eye droppers, as documented at https://www.accessibilityinsights.io/docs/windows/getstarted/colorcontrast/#manually-check-contrast-ratio. It's slower but should allow you to get accurate results until we have a better automated option in place.
From the triage discussion with the team, something to consider is that the new algorithm might impact performance. An option would be to create a toggle in Settings that allows users to try the new algorithm, however this might cause discoverability questions. Leaving this as ready for triage for more conversations next week
We've disabled automatic colour contrast analysis by default for now in latest main/next canary (see #1441) but keeping this open for tracking.
