Policy for minimum AT versions when adding plans to Test Queue

[jscholes]

Towards the end of the Community Group meeting today (March 20, 2025), we noted that test plans were being added to the Test Queue using NVDA 2020.4 as the minimum supported version for which we would accept results. At the time of writing, that version is over four years old, having been released on February 19, 2021[1].

With NVDA specifically, it seems unlikely that we would want to encourage test results being recorded with such an outdated version of a screen reader that is freely available. Indeed, other than corporate policies or the usage of an operating system that itself is extremely old, I would struggle to think of legitimate reasons for a tester to not be running the very latest. It's possible that a daily user of NVDA would want to hold off from updating because of e.g. add-on incompatibility, but that still usually leaves the option of using a portable version of the latest release for ARIA-AT testing.

That isn't necessarily the case for all other screen readers, though. For instance:

• With JAWS, licensing constraints may make it infeasible for someone to update immediately or at all.
• With macOS plus VoiceOver, the only way to obtain the latest screen reader is to update the entire OS, and someone may have valid reasons to not want to do that right away.

With all of that being said, I think the project would benefit from a general or per-AT policy around the minimum supported versions to specify when adding something to the Test Queue. I would encourage this to be informed by opinions from AT vendors where possible, as well as by the Community Group and other stakeholders.

CC @mcking65, @jugglinmike and @IsaDC

[1] https://www.nvaccess.org/post/nvda-2020-4/

[css-meeting-bot]

The ARIA-AT Community Group just discussed Issue 1213 - Minimum version requirements.

The full IRC log of that discussion

<jugglinmike> Topic: Issue 1213 - Minimum version requirements
<jugglinmike> github: https://github.com/w3c/aria-at/issues/1213
<jugglinmike> Matt_King: I don't know if we want a policy on this issue or if we just want it to be an admin decision every time you add something to the test queue
<jugglinmike> Matt_King: Depending on how the run is added, the app may or may not present a user interface for setting the version
<jugglinmike> Matt_King: I think that may be an omission in the design. I think addressing that omission may resolve the issue
<jugglinmike> IsaDC: That would definitely resolve it
<jugglinmike> Matt_King: We want to control the minimum version of the AT when adding a test plan to the test queue to the report status dialog
<jugglinmike> Matt_King: Carmen, can you create an issue for that in the aria-at-app repository?
<jugglinmike> Carmen: Sure

[ccanash]

Added app task https://github.com/w3c/aria-at-app/issues/1368 in case you want to close this @jscholes