nvda icon indicating copy to clipboard operation
nvda copied to clipboard

No result reported in calculator

Open CyrilleB79 opened this issue 2 years ago • 14 comments

Not to be confused with #15918

Steps to reproduce:

  • Open calculator
  • Type 1+1 on the numeric keypad.
  • Press NumpadEnter
  • Press NVDA+Tab

Actual behavior:

No result is reported when NumpadEnter is pressed.

Expected behavior:

The result should be reported with NumpadEnter.

NVDA logs, crash dumps and other attachments:

None

Notes

  • The issue seems to occur when doing an operation with the same one-digit number, e.g. 3-3, 6/6, 7*7
  • Does not seem to occur with 2-digit numbers or when the two operands are different.
  • Seems to be linked to the fact that the display was updated or not.
  • The issue is not present if you type the whole calculus on the main alpha-numeric keyboard.
  • The issue does not occur if you type 1+1 Enter such as the first 2 presses (1, +) are done on the numpad and the last two presses (1, Enter) are done on the main alphanumeric keyboard.

Maybe the issue is linked to the algorithm used to filter UIA notifications.

System configuration

NVDA installed/portable/running from source:

Installed

NVDA version:

2023.3

Windows version:

Windows 10 22H2 (AMD64) build 19045.3693

Name and version of other software in use when reproducing the issue:

CalculatorApp.exe

>>> focus.appModule.productVersion
'11.2307.4.0'

Other information about your system:

N/A

Other questions

Does the issue still occur after restarting your computer?

Yes

Have you tried any other versions of NVDA? If so, please report their behaviors.

No

If NVDA add-ons are disabled, is your problem still occurring?

Yes

Does the issue still occur after you run the COM Registration Fixing Tool in NVDA's tools menu?

Not tested

Cc @josephsl, @ruifontes

CyrilleB79 avatar Dec 14 '23 08:12 CyrilleB79

Hi,

A few things:

  • Is this happening with numlock disabled?
  • Is this specific to NVDA i.e. tested with Narrator as well?
  • Can you attach a debug log so we can see how NVDA treats numpad keys with numlock on?

Thanks.

josephsl avatar Dec 14 '23 14:12 josephsl

Testing with same version of Calculator, but on Windows 23H2 (Build 22631.2861) and NVDA alpha-30306,7ed382b5 (2024.2.0.30306) Using alphanumeric or numeric keyboard do not make any difference. Using any of them the result is not announced. Important part of log file: IO - speech.speech.speak (15:30:10.042) - MainThread (11472): Speaking ['Visualização é 0'] IO - braille.BrailleBuffer.update (15:30:10.043) - MainThread (11472): Braille regions text: ['Visualização é 0'] IO - inputCore.InputManager.executeGesture (15:30:35.024) - winInputHook (11680): Input: kb(desktop):numLockNumpad1 IO - speech.speech.speak (15:30:35.029) - MainThread (11472): Speaking [CharacterModeCommand(True), '1', EndUtteranceCommand()] IO - braille.BrailleBuffer.update (15:30:35.075) - MainThread (11472): Braille regions text: ['Visualização é 1'] IO - inputCore.InputManager.executeGesture (15:30:36.024) - winInputHook (11680): Input: kb(desktop):numLock+numpadPlus IO - speech.speech.speakTypedCharacters (15:30:36.043) - MainThread (11472): typed word: 1 IO - speech.speech.speak (15:30:36.044) - MainThread (11472): Speaking ['1'] IO - speech.speech.speak (15:30:36.044) - MainThread (11472): Speaking ['mais', EndUtteranceCommand()] IO - braille.BrailleBuffer.update (15:30:36.072) - MainThread (11472): Braille regions text: ['Visualização é 1 mais'] IO - speech.speech.speak (15:30:36.103) - MainThread (11472): Speaking ['Visualização é 1 mais'] IO - braille.BrailleBuffer.update (15:30:36.105) - MainThread (11472): Braille regions text: ['Visualização é 1 mais'] IO - inputCore.InputManager.executeGesture (15:30:36.704) - winInputHook (11680): Input: kb(desktop):numLockNumpad1 IO - speech.speech.speak (15:30:36.710) - MainThread (11472): Speaking [CharacterModeCommand(True), '1', EndUtteranceCommand()] IO - inputCore.InputManager.executeGesture (15:30:37.183) - winInputHook (11680): Input: kb(desktop):numpadEnter IO - speech.speech.speakTypedCharacters (15:30:37.193) - MainThread (11472): typed word: 1 IO - speech.speech.speak (15:30:37.193) - MainThread (11472): Speaking ['1'] IO - braille.BrailleBuffer.update (15:30:37.246) - MainThread (11472): Braille regions text: ['Visualização é 2']

As we can see, Braille announces correctly the result.

ruifontes avatar Dec 14 '23 15:12 ruifontes

  • Is this happening with numlock disabled?

The STR has no sense with numlock disabled, since you type the numbers and the operator on the numpad.

  • Is this specific to NVDA i.e. tested with Narrator as well?

Yes, the issue is specific to NVDA. Narrator behaves normally, i.e. without issue.

  • Can you attach a debug log so we can see how NVDA treats numpad keys with numlock on?

Here it is: nvda.log

CyrilleB79 avatar Dec 14 '23 15:12 CyrilleB79

@ruifontes wrote:

Testing with same version of Calculator, but on Windows 23H2 (Build 22631.2861) and NVDA alpha-30306,7ed382b5 (2024.2.0.30306) Using alphanumeric or numeric keyboard do not make any difference. Using any of them the result is not announced.

Note that the part of the logfile that you have pasted here is using the numlock. Could you paste the same using the alphanumeric keyboard? Which keyboard localization are you using (in case it depends on localization)

Note that I have tested with the following keyboard locales:

  • French (France)
  • English (United States)

CyrilleB79 avatar Dec 14 '23 16:12 CyrilleB79

Keyboard in pt_PT, Portuguese (Portugal) Log: IO - inputCore.InputManager.executeGesture (16:16:32.336) - winInputHook (11680): Input: kb(desktop):enter IO - speech.speech.speak (16:16:32.561) - MainThread (11472): Speaking ['Calculadora', CancellableSpeech (still valid)] DEBUG - NVDAObjects.NVDAObject._get_placeholder (16:16:32.565) - MainThread (11472): Potential unimplemented child class: <NVDAObjects.IAccessible.IAccessible object at 0x1371F5F0> IO - braille.BrailleBuffer.update (16:16:32.566) - MainThread (11472): Braille regions text: ['Calculadora'] DEBUGWARNING - RPC process 1228 (CalculatorApp.exe) (16:16:33.255) - Dummy-1181 (5008): Thread 8136, build\x86_64\remote\ia2Support.cpp, IA2Support_inProcess_initialize, 177: Not installing IA2 support as process is suspendable

IO - speech.speech.speak (16:16:33.350) - MainThread (11472): Speaking ['Calculadora', 'janela', CancellableSpeech (still valid)] IO - speech.speech.speak (16:16:33.368) - MainThread (11472): Speaking ['Visualização é 0', CancellableSpeech (still valid)] DEBUG - NVDAObjects.NVDAObject._get_placeholder (16:16:33.370) - MainThread (11472): Potential unimplemented child class: <NVDAObjects.IAccessible.IAccessible object at 0x1371F5F0> DEBUG - NVDAObjects.NVDAObject._get_placeholder (16:16:33.371) - MainThread (11472): Potential unimplemented child class: <NVDAObjects.UIA.UIA object at 0x118EAB10> DEBUG - NVDAObjects.NVDAObject._get_placeholder (16:16:33.371) - MainThread (11472): Potential unimplemented child class: <NVDAObjects.UIA.UIA object at 0x118F3E10> IO - inputCore.InputManager.executeGesture (16:16:38.248) - winInputHook (11680): Input: kb(desktop):1 IO - speech.speech.speak (16:16:38.253) - MainThread (11472): Speaking [CharacterModeCommand(True), '1', EndUtteranceCommand()] IO - braille.BrailleBuffer.update (16:16:38.285) - MainThread (11472): Braille regions text: ['Visualização é 1'] IO - inputCore.InputManager.executeGesture (16:16:38.799) - winInputHook (11680): Input: kb(desktop):plus IO - speech.speech.speakTypedCharacters (16:16:38.816) - MainThread (11472): typed word: 1 IO - speech.speech.speak (16:16:38.816) - MainThread (11472): Speaking ['1'] IO - speech.speech.speak (16:16:38.818) - MainThread (11472): Speaking ['mais', EndUtteranceCommand()] IO - braille.BrailleBuffer.update (16:16:38.833) - MainThread (11472): Braille regions text: ['Visualização é 1 mais'] IO - speech.speech.speak (16:16:38.866) - MainThread (11472): Speaking ['Visualização é 1 mais'] IO - braille.BrailleBuffer.update (16:16:38.868) - MainThread (11472): Braille regions text: ['Visualização é 1 mais'] IO - inputCore.InputManager.executeGesture (16:16:39.408) - winInputHook (11680): Input: kb(desktop):1 IO - speech.speech.speak (16:16:39.414) - MainThread (11472): Speaking [CharacterModeCommand(True), '1', EndUtteranceCommand()] IO - inputCore.InputManager.executeGesture (16:16:40.680) - winInputHook (11680): Input: kb(desktop):enter IO - speech.speech.speakTypedCharacters (16:16:40.684) - MainThread (11472): typed word: 1 IO - speech.speech.speak (16:16:40.684) - MainThread (11472): Speaking ['1'] IO - braille.BrailleBuffer.update (16:16:40.747) - MainThread (11472): Braille regions text: ['Visualização é 2'] IO - inputCore.InputManager.executeGesture (16:16:47.456) - winInputHook (11680): Input: kb(desktop):1 IO - speech.speech.speak (16:16:47.467) - MainThread (11472): Speaking [CharacterModeCommand(True), '1', EndUtteranceCommand()] IO - braille.BrailleBuffer.update (16:16:47.501) - MainThread (11472): Braille regions text: ['Visualização é 1'] IO - inputCore.InputManager.executeGesture (16:16:47.903) - winInputHook (11680): Input: kb(desktop):plus IO - speech.speech.speakTypedCharacters (16:16:47.922) - MainThread (11472): typed word: 1 IO - speech.speech.speak (16:16:47.923) - MainThread (11472): Speaking ['1'] IO - speech.speech.speak (16:16:47.923) - MainThread (11472): Speaking ['mais', EndUtteranceCommand()] IO - braille.BrailleBuffer.update (16:16:47.952) - MainThread (11472): Braille regions text: ['Visualização é 1 mais'] IO - speech.speech.speak (16:16:47.989) - MainThread (11472): Speaking ['Visualização é 1 mais'] IO - braille.BrailleBuffer.update (16:16:47.992) - MainThread (11472): Braille regions text: ['Visualização é 1 mais'] IO - inputCore.InputManager.executeGesture (16:16:48.384) - winInputHook (11680): Input: kb(desktop):2 IO - speech.speech.speak (16:16:48.389) - MainThread (11472): Speaking [CharacterModeCommand(True), '2', EndUtteranceCommand()] IO - braille.BrailleBuffer.update (16:16:48.434) - MainThread (11472): Braille regions text: ['Visualização é 2'] IO - inputCore.InputManager.executeGesture (16:16:48.816) - winInputHook (11680): Input: kb(desktop):enter IO - speech.speech.speakTypedCharacters (16:16:48.820) - MainThread (11472): typed word: 2 IO - speech.speech.speak (16:16:48.821) - MainThread (11472): Speaking ['2'] IO - braille.BrailleBuffer.update (16:16:48.886) - MainThread (11472): Braille regions text: ['Visualização é 3'] IO - speech.speech.speak (16:16:48.921) - MainThread (11472): Speaking ['Visualização é 3'] IO - braille.BrailleBuffer.update (16:16:48.924) - MainThread (11472): Braille regions text: ['Visualização é 3']

ruifontes avatar Dec 14 '23 16:12 ruifontes

Hi,

Affects Narrator as well. Here's why:

  1. User enters "1": UIA notification event is fired.
  2. Then user enters "+": UIA notification event is fired.
  3. User enters "1": UIA notification event is NOT fired.
  4. User presses Enter: UIA notification event is fired.

NVDA does handle UIA notification event. It's just that it doesn't know that a number key was entered prior to Enter being pressed, so the "no results announcement" state is not toggled correctly. However, as the above process shows, the fundamental issue is with Calculator itself not firing notification event if both operands around an operator are single digits. This problem is "resolved" if more than two operands are present.

We can solve this in a number of ways:

  1. Handle ALL UIA notification events although it will cause regressions whee expressions are announced while speak typed characters is on.
  2. Inform results script (Enter/Numpad Enter/Escape) to inform Calculator app module to announce results no matter what.
  3. Send feedback to Microsoft Calculator team (they have a GitHub repo) and ask them to look into UIA notification event logic from their side.

Of these, I advise the third option, as options 1 and 2 can cause regressions and amount to fixing the problem at the surface level a.k.a. workaround, respectively.

Postscript: I won't respond to this issue until next week at the earliest as I'm taking finals and to look into this matter from different angles.

Thanks.

josephsl avatar Dec 14 '23 17:12 josephsl

@josephsl, as written in https://github.com/nvaccess/nvda/issues/15923#issuecomment-1856110156, I have not the problem with Narrator. On your side, you write:

Affects Narrator as well.

Does it mean that you do not get the correct result with Narrator on your side?

CyrilleB79 avatar Dec 14 '23 22:12 CyrilleB79

Hi,

What I looked at is not just the result of what Narrator does (it does announce results, by the way), but the events that led to Narrator not announcing single digits between operators, which turned out to be UIA notification event problem from Calculator app itself. I think this is one of those cases where resolving it from screen readers is a short-term solution, with the actual issue coming from the app. Short of asking Calculator team (Microsoft) to look into this matter, if you do want NVDA side solution, please let us know.

Thanks.

josephsl avatar Dec 15 '23 20:12 josephsl

Since you have identified an issue on the app side, I agree that it should be reported to Microsoft. Given you have the evidences that the app does not behave correctly, would you be able to report it to them?

If you cannot, e.g. for lack of time, no worry. Just indicate it here as a status.

Though, if possible, it would be nice to have a fix in NVDA in the meantime, given we do not know how much time it will take to Microsoft to fix the problem.

CyrilleB79 avatar Dec 15 '23 21:12 CyrilleB79

See microsoft/calculator#2097.

josephsl avatar Dec 15 '23 21:12 josephsl

Hi,

As a temporary measure, I will include a fix for this issue in Windows App Essentials 24.01 until Microsoft fixes this.

Thanks.

josephsl avatar Dec 19 '23 23:12 josephsl

@msftedad can you take this up with the development team please? It needs an external fix from microsoft side, particularly firing the correct UIA events.

Adriani90 avatar Aug 21 '24 07:08 Adriani90

@Adriani90 - thank you for tagging us here, we will discuss and share this MS calculator Team and inform you with progress here.

msftedad avatar Aug 22 '24 03:08 msftedad

@Adriani90 - This case is forwarded to MS calculator team.

msftedad avatar Aug 23 '24 06:08 msftedad

@CyrilleB79 Is this issue also occuring on Windows 11 24 H2 last stable update with NVDA 2025.1 Beta?

Adriani90 avatar May 22 '25 22:05 Adriani90

I am not running Windows 11 machines yet. I will do so in the seconde part of 2025. Please ping me again when Windows 10 is out of support.

CyrilleB79 avatar May 23 '25 06:05 CyrilleB79

This is still happening with Calculator 11.2502.2.0 and NVDA alpha-36575,7191f78e (2025.2.0.36575) on Windows Version 24H2 (OS Build 26100.4061) x64

SaschaCowley avatar May 30 '25 00:05 SaschaCowley