Unable to use applications that hook the arrow keys using Windows Console Host.

[TheQuinbox]

Windows Terminal version

1.14.1962.0

Windows build number

Windows 10 21H2 (64-bit) build 19044.1826

Other Software

GitHub CLI latest version.

Steps to reproduce

1. Run a GH (or other application that hooks the arrow keys) command
2. Attempt to arrow.

Expected Behavior

This should behave exactly as it does in cmd. I'm able to arrow through the options accessibly and select an option.

Actual Behavior

When using both Narrator and NVDA, I hear "blank". I'm unable to confirm if this is a visual problem as well.

[TheQuinbox]

I can now confirm that its a screen reader only problem. The cursor moves, but NVDA and Narrator just don't read it

[zadjii-msft]

@carlos-zamora You know if we've got this one on the backlog? You're usually faster at searching the a11y threads than me ☺️

[carlos-zamora]

Doesn't sound familiar from the backlog. This sounds pretty sever though. @TheQuinbox a few questions for you:

• What shell are you using? It sounds like you're using PowerShell, but can you tell me the version? FYI "PowerShell 7.3.0-preview.6" adds suggested text as you type. I still don't get the "blank" behavior, but idk if that's related?
• When you say "attempt to arrow", what direction are you going in? up/down to select the previous/next command? Or left, right to move the cursor?
• What GH command are you executing exactly? I can't find any that hook the arrow keys. Upon execution (at least the ones I use), the command just outputs text that is then read by the screen reader, as expected.

I can't get a repro, so I'm hoping some more details can help. Thanks!

[TheQuinbox]

Hi @carlos-zamora, I'm using Command Prompt, but I just tested, and it also happens with PowerShell. I'm using the up and down arrow keys to move between options. If using NVDA's review cursor, I can tell what option is selected, but it takes significantly more time. I'm running:

$ gh pr create

when on a branch with uncommitted changes. I'm then brought to a wizard that asks me to select an option.

Its probably worth mentioning that I've also tested this in standalone CMD.exe and PowerShell.exe shells and it doesn't happen.

If still unable to reproduce, would an audio demonstration possibly help?

Thanks so much for looking into this! :)

[carlos-zamora]

Awesome. Got it! Thanks!

For visual users, this is what the UI looks like:

In this case, the text reads:

D:\Projects\terminal>gh pr create
? Where should we push the 'dev/cazamor/test' branch?  [Use arrows to move, type to filter]
  microsoft/terminal
> Create a fork of microsoft/terminal
  Skip pushing the branch
  Cancel

The key thing to see here is the ">". Pressing up and down "moves" that symbol.

Here's the tricky party though, when up/down is pressed, the screen reader reads the entire output again, suggesting that gh.exe is actually deleting and re-rendering the entire output again with the ">" in the correct place.

Now, as for a fix, I'm a bit stumped. I'll throw the "Needs-Discussion" tag so that we discuss this at an upcoming sync. Fair warning, I'm about to head out on vacation for a bit, so it'll probably be a while before I get back to you on this one. Maybe 2 weeks? Don't worry though, as long as that "Needs-Discussion" tag is on the issue, we'll get to it (we go over as many as we can when we meet up at our weekly sync meeting)

[TheQuinbox]

Awesome, sounds great! I have honestly no idea how Terminal renders, but I do remember hearing that it uses the GPU. Maybe that's playing a part? As stated previously this doesn't happen anywhere else.

Thanks for looking into this! :)

[TheQuinbox]

Hi, have there been any updates on this issue? Am still fully able to reproduce on Windows 11 22h2 (build 22000.198), with Windows Terminal 1.15.2875.0. Thanks!

[carlos-zamora]

Hi @TheQuinbox. Sorry I didn't get back to you! Yeah, so like I said in my comment above, I don't think there's a quick-n-easy fix for this. gh.exe is rewriting the entire prompt every time, resulting in the screen readers thinking either (1) the entire prompt is "new content" that needs to be reread, or (2) nothing changed because the new content is largely similar to the old content.

You could file a bug on NVDA/Narrator to have better handling for this kind of scenario. That might be the quickest way forward to see change here.

Outside of that, the best idea I've come up with is to develop a system for which command line applications can explicitly tell screen readers what to read out. However, this involves changes on our end and on the app's end (in this case, gh.exe), and that'll take months to get done.

Regardless, I'm sorry to say that implementing a solution here is going to take a long time. I'll keep this thread in the loop as new updates develop.

[zadjii-msft]

x-linking:

• Control Screen Reader from Applications.
  • This is a good example of how we might want to enable this in the future. It would require apps to build in support for this manually, but diff'ing or trying to do heuristics is probably a non-starter
• https://github.com/microsoft/winget-cli/issues/161
  • We've talked with the winget folks and they're very interested in this.