MenuStrip accessibility for screen readers

[Menelion]

.NET version

6.0

Did it work in .NET Framework?

No

Did it work in any of the earlier releases of .NET Core or .NET 5+?

N/A

Issue description

The Story

Before, in .Net Framework, there was a classic MainMenu control that provided a classic native menu bar. It was super accessible, all screen readers read it correctly, the Escape key was processed automatically, as was the Alt key (it toggled the focus between the menu bar and the main window). to put it short, it was analogous to a menu bar written in pure Windows API.

However now, in .Net 6, there is only this MenuStrip class which behaves differently:

• JAWS for Windows reports each item of the horizontal menu bar as having a submenu. Each vertical dropdown menu, when entered, is reported as context menu which is wrong. NVDA reports those items as collapsed, which is also wrong, they are just menu bar items. Narrator is super verbose in any menu bar, so I would suggest to try the other two.
• The Escape key seems to have effect but the screen readers do not report "leaving menus" and the menu status seem to persist. At least, JAWS reports "leaving menus" only when alt-tabbing out of the sample application. The Alt key seems to have no effect.
• The checked status of a checkable menu item is reported only by Narrator. As it is reported correctly in native menu bars, the issue is in the custom control itself.
• The menu bar implemented this way lacks the system menu with elements like Move, Size, Minimize, Maximize etc. Alt+Space also does not work. With the native menu bar this menu is given free of charge.

I sketched a basic sample to show what I mean.

Steps to reproduce

1. Run the sample app.
2. Launch a screen reader, for instance, JAWS for Windows.
3. Press Alt to go to the menu bar and see the behavior outlined in the issue description.
4. Run Visual Studio any version, 2022 Preview included.
5. Press Alt to go to the menu bar and spot the differences.

[merriemcgaw]

@dmitrii-drobotov please have someone on the team investigate.

@Menelion thank you for the detailed description of what you're seeing in the MenuStrip control! Accessibility is a top priority for the team and we strive to bring full accessibility to every control we provide. We'll investigate and make sure that we report the correct UIA tree for all scenarios. We'll also take a look and see if JAWS and NVDA are implementing workarounds to get around old bugs that we no longer have in the newer controls. We've been working to go beyond the legacy accessibility implemented in the Win32 APIs and make sure we have full UIA support across the MenuStrip (and ToolStrip). If we find that NVDA or JAWS needs to take fixes to work with UIA Accessibility implementation, we'll make sure we get those filed.

Adding the System Menu back to WinForms windows is a great idea, and we'll take that up as a separate investigation/feature request. Thank you for bringing this issue to our attention. We will be getting as much of it fixed as possible for .NET 8 (our next release). If you are blocked for something in your .NET 6 application, we can consider taking individual fixes as a servicing fix in .NET 6, though we have to be mindful of what we take in a servicing release.

[merriemcgaw]

@Tanya-Solyanik FYI

[Menelion]

@merriemcgaw Thank you for such a prompt and uplifting reply! Can I help you somehow? I'd love to contribute (planning to clone the repo and see the code), so would you encourage investigation from my part? Also, I'm a private beta tester for Vispero (JAWS for Windows) and have contacts there, that can also be of help probably. I know that Vispero has a partnership with Microsoft, but still maybe I can be useful. Thank you!

[Menelion]

Just thoughts and some stupid questions, sorry, I'm totally new in contributing to .Net and actually not so much experienced in development for it either 😊

• Looking at the code, it seems that the Alt key processing for focusing and unfocusing the MenuStrip is actually there, but for me it doesn't work correctly, or as correctly as it does for the native menu bar (MainMenu in .Net framework version of WinForms). I don't however see any notions of Escape key processing, is it missing or is it implied somehow?
• The System menu is invoked (via interop) by Alt+Space, and it does work indeed. The only place where it's absent is the menu bar itself.

My main question is: if I build WinForms from sources, make some changes to the accessibility objects representation, how do I target my sample app to run against this local build, so I see what screen readers output with my future changes? Thank you in advance!

[merriemcgaw]

I'm so excited to have a community member passionate about accessibility contributing! @dmitrii-drobotov and @Tanya-Solyanik are great sources of information and ideas. To get up and running I'd suggest starting with our Developer Guide. The whole WinForms community will be happy to help answer any questions you have as you are working in the repo.

[dmitrii-drobotov]

@Menelion thank you for a detailed report and providing a sample application! We'll certainly investigate the issue.

if I build WinForms from sources, make some changes to the accessibility objects representation, how do I target my sample app to run against this local build, so I see what screen readers output with my future changes? Thank you in advance!

Please refer to the Debugging documentation page. There are two ways you could achieve it — either replace the system-wide .NET SDK or add references to the local build in your project.

[Menelion]

One more finding after deeper inspection with JAWS: MenuStrip does not generate the "leaving menus" event. Sorry, I don't know yet the exact name of the event in UIA, but it's the one it should generate when focus leaves the menu. So what happens: if we press a standalone Alt key, it toggles between the menu bar and the main window, which is correct. But the problem is that JAWS doesn't know that the menu has been left, so its behavior is as if it still was in the menu mode. The only workaround is to alt-tab to another application, then you will hear "Leaving menus" and the menu state will be dismissed.

[Tanya-Solyanik]

@Menelion - I tried out ToolStrip with a drop down button on .NET Framework4.8.1 and JAWS version 2022.2207.25 ILM and I got the "leaving menu" announcement. Please share your code.

ToolStrip code:

            // 
            // toolStrip1
            // 
            this.toolStrip1.ImageScalingSize = new System.Drawing.Size(40, 40);
            this.toolStrip1.Items.AddRange(new System.Windows.Forms.ToolStripItem[] {
            this.toolStripDropDownButton1});
            this.toolStrip1.Location = new System.Drawing.Point(0, 0);
            this.toolStrip1.Name = "toolStrip1";
            this.toolStrip1.Size = new System.Drawing.Size(800, 51);
            this.toolStrip1.TabIndex = 0;
            this.toolStrip1.Text = "toolStrip1";
            // 
            // toolStripDropDownButton1
            // 
            this.toolStripDropDownButton1.DisplayStyle = System.Windows.Forms.ToolStripItemDisplayStyle.Image;
            this.toolStripDropDownButton1.DropDownItems.AddRange(new System.Windows.Forms.ToolStripItem[] {
            this.item1ToolStripMenuItem,
            this.item2ToolStripMenuItem});
            this.toolStripDropDownButton1.Image = ((System.Drawing.Image)(resources.GetObject("toolStripDropDownButton1.Image")));
            this.toolStripDropDownButton1.ImageTransparentColor = System.Drawing.Color.Magenta;
            this.toolStripDropDownButton1.Name = "toolStripDropDownButton1";
            this.toolStripDropDownButton1.Size = new System.Drawing.Size(66, 44);
            this.toolStripDropDownButton1.Text = "toolStripDropDownButton1";
            // 
            // item1ToolStripMenuItem
            // 
            this.item1ToolStripMenuItem.Name = "item1ToolStripMenuItem";
            this.item1ToolStripMenuItem.Size = new System.Drawing.Size(448, 54);
            this.item1ToolStripMenuItem.Text = "item1";
            // 
            // item2ToolStripMenuItem
            // 
            this.item2ToolStripMenuItem.Name = "item2ToolStripMenuItem";
            this.item2ToolStripMenuItem.Size = new System.Drawing.Size(448, 54);
            this.item2ToolStripMenuItem.Text = "item2";

Accessibility Insights reports: StructureChanged - tool bar 'toolStripDropDownButton1DropDown' when I press the ALT key to leave the drop down

For the mainMenu AccessibilityInsights report AutomationFocusChanged window 'Form1'

[Menelion]

@Tanya-Solyanik Have you tried a MenuStrip set as MainMenuStrip for the form? See my sample app.

[Tanya-Solyanik]

@Menelion - in my experiments on .NET 7, it does not matter if MenuStrip is the MainMenuStrip on the form. However, presence of other controls makes the difference. I assume that screen readers do message throttling (Narrator does). If user actions come in faster that announcements can be read, some announcements are dropped. If there are other focusable controls on the form, JAWS reads their names after the user presses ALT because JAWS does not have time to read the "leaving menu" announcement.

[Menelion]

@Tanya-Solyanik Oh no, message throttling is not the reason. The problem is JAWS behaves differently in so-called menu mode, i.e., when a menu is active. the simplest way of checking this is alt-tab out of the app and back. If the menu was stuck, JAWS would say "leaving menus" only after alt-tabbing out of the app.

Anyway, how can I retarget to .Net 7? I'll try to build from main and add it as a reference for a clearer experiment because now I'm on .Net 6.

[Tanya-Solyanik]

I'm pretty sure target framework does not matter. Official download - https://dotnet.microsoft.com/en-us/download/dotnet latest builds - https://github.com/dotnet/installer#installers-and-binaries - do not install 8 yet, as you will not be able to target it. I also run the latest VisualStudio 2022 IntPreview, but this is not a requirement, any version of VS 2022 will work. To retarget - set TargetFramework property in the project file - <TargetFramework>net7.0-windows</TargetFramework>. Or change target framework property in the project property pages in the VS (Alt+Enter when focus is on the project in solution explorer).

By menu mode, do you mean menu bar has focus?

[Menelion]

Sorry, asked too hastily, managed to retarged to .Net 7 without issues.
In JAWS there is a built-in function GetMenuMode which deals with menu events like SYSTEM_EVENT_MMENUSTART, SYSTEM_EVENT_MENUEND and alike for menu pop-ups. Unfortunately the code of the function is buried in the closed part of JAWS's sources, so I cannot inspect it deep enough. I'm talking about these native events. there are their UIA counterparts, and the MenuStrip control for some reason does not properly fire events for leaving menus. Speaking of JAWS, it behaves totally differently when in menu mode: character, word and line reading act differently, keyboard commands are restricted and so on.

[v-elnovikova]

The checked status of a checkable menu item is reported only by Narrator. As it is reported correctly in native menu bars, the issue is in the custom control itself.

Menu item checked state is reported in legacy MSAA State value only. Created issue #8307 to add UIA Toggle pattern support.

[v-elnovikova]

JAWS for Windows reports each item of the horizontal menu bar as having a submenu.

Cannot reproduce this behavior. Both Narrator and NVDA announce "Just menu bar item" (i.e. item name) if item has no submenu. Otherwise, word "collapsed" is added, for example "File, collapsed". JAWS uses word "submenu" in this case, for example "File, submenu". Tested on JAWS 2022.2110.60, NVDA 2022.3.2, Narrator on Windows 11.

@Menelion Could you please provide the details of your environment - OS version, .NET runtime and/or SDK version? What version of JAWS and NVDA are you using? Do you have any custom Scripts in JAWS?

[Menelion]

@v-elnovikova Thank you for coming back to this issue. the behavior you're describing ("submenu" in JAWS and "collapsed" in the other two) is incorrect. It's not how a normal menu bar should be processed. Again, please load VS 2022, open its menu bar and listen to what it says. That's how it should be. Plus please see my ramblings about the leaving menus event in the comments, maybe it's also a rather quick fix. Thank you in advance! P.S. If it's easier if I show you what I mean, please tell me how we can connect.

[v-elnovikova]

@Menelion For me, menu bar in VS 2022 (version 17.4.3 Community) is also announced with "submenu" or "collapsed" words, the same as MenuStrip in your sample WinForms app. Here is the output I get for JAWS 2022 in VS menu:

TextFile1.txt - Microsoft Visual Studio TextFile1.txt Tab
No issues found Ln: 1 Ch: 1 Text Editor Menu bar File To navigate press Left or Right Arrow. F menu
New sub menu To move through items press up or down arrow. N Open sub menu O Clone Repository... E Go To sub menu G Leaving menus TextFile1.txt Tab

And for WinForms app:

Newer Menu Bar Example Input Text: Edit Type in text. Alt+t Menu bar File sub menu To navigate press Left or Right Arrow. f menu New... Ctrl+N To move through items press up or down arrow. n Open... Ctrl+O o Save Ctrl+S s Undo Ctrl+Z u Context Menu Edit To navigate press Left or Right Arrow. leaving menu bar

There is "sub menu" word in both cases, so it's hard to tell what needs to be changed in WinForms. Or do you mean that for top-level items (in VS it's File, Edit, View and so on) it doesn't say "sub menu" in VS and this is what we should remove on WinForms side? Could you please share what JAWS says in VS menu in your case? It would help us determine the expected result.

As for leaving menu event, we are still looking into it and will keep you updated.

[Menelion]

@v-elnovikova I get it's quite hard to grasp, so thank you for your patience.

I'm talking about navigating horizontally in the menu bar, without going down the menus. So, if you do Right Arrow straight away from the File menu, you'll hear in VS: File, Edit, View, etc., no "submenu" or "collapsed" marks. It is correct on items like "new" though because they do have submenus and they are a part of a vertical menu, it's a separate thing. So, to do what I would like you to hear, press alt and then immediately go right with your Right Arrow in both cases, you'll spot the difference.

Please feel free to ask further if it's not comprehensible. Thanks!

[dmitrii-drobotov]

@Menelion - to track current progress on the issue, we've added check boxes and links to work items for each of the problems in the description.

Also, to clarify the following item:

NVDA reports those items as collapsed, which is also wrong, they are just menu bar items. Narrator is super verbose in any menu bar, so I would suggest to try the other two.

This seems like an expected behavior because it matches how NVDA and Narrator often read other menus, for example in Visual Studio. It comes from implementing ExpandCollapse UI Automation pattern, which we do according to the documentation.

[SimonZhao888]

Hi @Tanya-Solyanik, because we had closed issue 8326, which is related to so we want to know if this issue needs to be fixed or not.

[Menelion]

I'm sorry @SimonZhao888, I'm just the reporter, not a Microsoft employee, but I can confirm that it works decently in .NET 8. That was a great improvement from your part, and I'm infinitely grateful as now we are free to develop our blind-related projects. Thank you!