Screen-reader accessibility in html report

[vincent-lg]

coverage is a great tool, complementing unit testing to create more stable projects. Developers using screen-readers, like myself, might have some difficulty in trying to improve coverage of their project.

Problem

Although html is a detailed report, its accessibility with a screen-reader could be improved without so much effort. Alternatively, to have these information in console, for instance, could also be extremely useful, but this issue focuses on the HTML report.

Currently

The html main report (the one you would access through the index.html file) is accessible as-is. Things get a bit more complicated when opening the report of a specific file:

• I can see the coverage summary: "Coverage for context\base.py : 28%", then a bit below "79 statements   22 run 57 missing 0 excluded".
• I can see the individual lines of the file with line numbers.
• I can't easily see which line / statement is tested (covered) and which one is not.
• It seems clicking on the buttons ("79 statements", "22 run", "57 missing", "0 excluded") doesn't do anything, although it might do so, visually.
• It seems I can't click on the "short shortcut keyboard" image. The screen-reader sees it as an image, clicking on it doesn't display anything. Note that screen-readers support keyboard shortcuts of their own, and shortcuts that conflict with screen-readers might be hard to get around for users.

What I suggest

Line information

The most important issue is the impossibility to know what a line is: covered, not covered, not important. This is presumably done with colors, and the screen-reader can access colors, but you need to request colors on any particular line. Which doesn't make the experience that fluid. Tested with NVDA (Non-Visual Desktop Access), when pressing NVDA + F to see the color information on a line, I get something like this:

Consolas, 11pt, light grey on white, baseline, align right

As you can imagine, inferring what this means can be tricky and, anyway, there's a lot of information that's not exactly needed.

What to do? There are a few possibilities: adding a text (or even a symbol) only the screen-reader perceives and that is not displayed to most users. Bootstrap supports it with the "sr-only" class for instance. I'm not sure what you're using. This symbol (or label) could be inserted before line numbers, like this:

Missing : 322, def call_in(seconds, callable, *args, **kwargs): 

Alternatively, instead of a clear label, this could be a short symbol, something straight-forward enough:

m322, def call_in(seconds, callable, *args, **kwargs): 

("m" standing for missing, a dot could stand for covered, an X for excluded, and so on.)

Filters

It might also be good to have the buttons above the report act as filters. If you click on "57 missing", for instance, then only missing statements would appear, excluded and not-run lines would just disappear. As stated above, this is not the case, though it might be visually. The screen reader doesn't see any difference, so clicking on these buttons doesn't accomplish much.

Keyboard shortcuts

It might be worth placing keyboard shortcuts on these buttons, if they behave like filters: number of statements, number of run statements, number of missing statements, number of excluded statements. If so, I recommend using the browser-compatible shortcuts on buttons and advise against coding such a thing in JavaScript.

Why? As stated, the screen-reader has its own shortcuts. For instance, pressing "h" will jump to the next header on the page. Almost every single key is linked to the screen-reader, and by default, most screen-readers only pass these shortcut keys if they don't map anything in their features. On the other hand, browser-compatible shortcuts always are safe from conflict. In Firefox for instance, adding a shortcut tag to a button enables the Alt + Shift + key shortcut, which is always free for screen readers. Besides, because it remains plain HTML, screen readers can often run a check on these shortcuts and see if they would pose an issue. Therefore, I advise using the HTML tags to link a key with a button and leaves the rest to the browser. Yeah, sometimes, best to trust the browser!

All this is recommendation of course, and I tried to detail it as much as possible, considering web accessibility can be tricky and often appears more complicated than it really is. I'm aware that this might need further discussion, and please don't hesitate to ask more questions. This post is already long and I don't wish to flood with unwanted information.

Thanks for the great work,

[nedbat]

@vincent-lg thanks for the detailed information. I'm familiar with these ideas, but have not done a good job carrying them through into the coverage report.

Definitely we can add information to each line for the screen reader. I welcome your recommendations for what would work best. Filtering lines is a thornier issue. The buttons at the top do filter the lines, but by changing their background color, not by removing them completely.

Can you say more about the browser-compatible shortcuts for the buttons? Google isn't helping me find the thing you seem to be talking about.

[proudfeet]

I was actually about to raise an issue highlighting some of the accessibility points on the HTML form identified by a colleague but think they are mostly addressed above. Appreciate it is a few years old now.

I would be interested in helping contribute some of the components mentioned above.

Would it be sensible to try and split out a few individual but linked accessibility issues from the above?

[nedbat]

Thanks, I appreciate any help. If you have solutions, we don't need to make separate issues, the pull requests will be a fine start.