Pressable component's 'onPressIn/Out' don't work on scan mode

[axelfratoniMS]

Problem Description

If we create a Pressable or a TouchableWithoutFeedback component and we activate Narrator on scan mode (ctrl + win + enter to activate Narrator, caps + space to activate scan mode), the callback functions on the component stop getting called when focusing with Tab and activating with Enter/Space.

Example:

<Pressable
      accessible={true}
      focusable={true}
      onPress={handleOnPress}
      onPressIn={handleOnPressIn}
      onPressOut={handleOnPressOut}
>

If we activate the narrator and navigate to this component using Tab, when we press Enter or Space we can verify that handleOnPress, handleOnPressIn and handleOnPressOut get called. If we activate Scan Mode and press Enter or Space, none of these functions get called.

Steps To Reproduce

1. Create a React Native Windows app.
2. Create a Pressable or TouchableWithoutFeedback component as described in Problem Description.
3. Activate the Narrator
4. Activate Scan Mode
5. Navigate to the created component by pressing Tab
6. When focusing the component, press Enter or Space to activate
7. Verify that none of the callback functions get called

Expected Results

When using Narrator on Scan Mode, the component's behavior should be the same as with Scan Mode off.

CLI version

6.4.0

Environment

System:
    OS: Windows 10 10.0.19044
    CPU: (12) x64 Intel(R) Xeon(R) W-2235 CPU @ 3.80GHz
    Memory: 30.72 GB / 63.73 GB
  Binaries:
    Node: 16.14.0 - C:\Program Files\nodejs\node.EXE
    Yarn: 1.22.19 - ~\AppData\Roaming\npm\yarn.CMD
    npm: 8.3.1 - C:\Program Files\nodejs\npm.CMD
    Watchman: Not Found
  SDKs:
    Android SDK: Not Found
    Windows SDK:
      AllowDevelopmentWithoutDevLicense: Enabled
      Versions: 10.0.18362.0, 10.0.19041.0
  IDEs:
    Android Studio: Not Found
    Visual Studio: 16.11.32602.291 (Visual Studio Enterprise 2019)
  Languages:
    Java: Not Found
  npmPackages:
    @react-native-community/cli: Not Found
    react: Not Found
    react-native: Not Found
    react-native-windows: Not Found
  npmGlobalPackages:
    *react-native*: Not Found

[agniuks]

Was able to repro on gallery. Works when activating through mouse, but not with keyboard. Looks as though hitting enter/space isn't being registered at all.

[agniuks]

Hey @axelfratoniMS, could you try adding accessibilityRole="button" and try again? I'm able to get onPress working that way. For example, this works as expected in scan mode (although onPressIn/onPressOut are still not) :

[axelfratoniMS]

@AgneLukoseviciute adding the accessibility role fixes onPress but doesn't fix onPressIn and onPressOut. Is this also what you saw?

[agniuks]

@axelfratoniMS yeah, same for me. I'll look into onPressIn/Out, could be something weird about how the Xaml we use there operates in scan mode.

[agniuks]

Needs more investigation, moving to .71

[YajurG]

Added some log and debug statements to compare onPress and onPressIn/onPressOut, the component is not being called when it's set to onPressIn or onPressOut (nothing is logged), but the log statement is hit with onPress and the count keeps incrementing. Would next step be to look at underlying XAML code?

[agniuks]

@YajurG I would look at the Pressable and Pressability files. Can you repro that this is only an issue with keyboard and not mouse? I see a comment on one of those files that may be of interest: https://sourcegraph.com/github.com/microsoft/react-native-windows/-/blob/vnext/src/Libraries/Pressability/Pressability.windows.js?L630

[YajurG]

It is a keyboard only issue - when I click on the Pressable in scan mode the logging statements are hit and the code runs

[agniuks]

It is a keyboard only issue - when I click on the Pressable in scan mode the logging statements are hit and the code runs

yeah, so given that and the linked comment, there might be something wonky about keyboarding/key events. May be a regression, maybe it was never tested in scan mode. Would be worth investigating :) One thing to try would be seeing how that code flows (pressability, pressable.js) when in scan mode vs when not, and when using mouse vs keyboard, might reveal something helpful.

[shreyas-algo]

Can we prioritize this fix please? It is being flagged as a sev2 from Accessibility teams.

[Yajur-Grover]

This issue is a priority and is being actively worked on. Update from recent testing- when in scan mode, using Ctrl+Enter or Ctrl+Space (vs just Enter or Space) invokes the onPressIn/onPressOut handlers. The cause of this behavior is unclear, and I have looped in the Narrator Dev team for their insight into possible causes.

[chrisglein]

@AgneLukoseviciute adding the accessibility role fixes onPress but doesn't fix onPressIn and onPressOut. Is this also what you saw?

We'll need to get to the source of this repro to understand more of why this is important. The purpose of onPressIn and onPressOut are to track the temporary pressing state. For example, if your mouse button is depressed and you are hovering over a button, you should get an onPressIn. When you hover off the button while still holding the mouse button, you should get an onPressOut. You should only trigger action on onPress for this reason; onPressOut does not indicate user intent to click the button.

Given that when the screen reader is active in scan mode it's not direct user input to the app (it's filtered through the screen reader), we may not make the same assumptions about those states being triggered (similar to if you use the accessibility invoke pattern). It sounds like the behavior we're seeing here is by design, but we need to understand more about the app's usage in case we're misunderstanding.