Add long tap to right click support

[frenzibyte]

• Closes #5200

Went with a hold delay of 500ms from real device testing, feels pretty well. The implementation behind this was to trigger a right click and "cancel" the held left button (in which it would be released but not trigger a click or otherwise).

In addition, I had to move the button input application to happen only if e.IsActive != null rather than always, since the latter would cause left clicks when moving a touch slightly after triggering a right click.

[peppy]

To make this feel good, we're going to need some kind of visual feedback during the operation. I'm not yet sure where this should be done, but may be good to do some initial experimentation and see what you come up with.

Take a look at iOS for the best implementation I've seen to date (actually scaling the target element during the operation). I'm not sure how possible this is to do for us since we haven't really thought about it from the outset. Alternatives would be to show some kind of overlay effect on or under the cursor (windows does this) which scales or radial fills to show how far through the hold-to-right-click cycle the user is.

I can see this being something game/applications may want to override (ie. osu! side making it part of the menu cursor seems like a good place), so a good start could be exposing this from InputManager as a progress bindable, or at very least Started and Ended events along with touch_right_click_delay being public.

[bdach]

Looking at the implementation from the technical side, I'm not sure I agree with the way these "long press" events are synthesised from normal touches. Both iOS and Android have dedicated OS pathways for this, and I'm not sure we should be doing our own things rather than using what the OSes provide.

[frenzibyte]

Looking at the implementation from the technical side, I'm not sure I agree with the way these "long press" events are synthesised from normal touches. Both iOS and Android have dedicated OS pathways for this, and I'm not sure we should be doing our own things rather than using what the OSes provide.

The one concern I have is we use this gesture whenever touch input is not handled by the held drawable, rather than always using it.

If we were to implement this from OS events it would mean defining the concept of a “gesture” to the o!f input flow until it reaches InputManager where we can add visual feedback to it.

I believe it’s better implementing it this way until we require long touch gestures in game components that handle multi-touch input (and even then I would still imagine using Scheduler in some form of extension method or something like the pre-existing RepeatingButtonBehaviour would be just enough)

[bdach]

If we were to implement this from OS events it would mean defining the concept of a “gesture” to the o!f input flow until it reaches InputManager where we can add visual feedback to it.

This is a hypercorrect stance. From a pragmatic standpoint I see no reason to take it this far. You could still proxy the OS "long press gestures" to right clicks for the time being, but the OS would be responsible for actually detecting them.

The reason why I'm not really all that okay with the current proposal is that each OS has its own paradigms and if the OS provides an input flow for long press, we should use that to be consistent with literally everything else on the platform if it's feasible.

[frenzibyte]

If we were to implement this from OS events it would mean defining the concept of a “gesture” to the o!f input flow until it reaches InputManager where we can add visual feedback to it.

This is a hypercorrect stance. From a pragmatic standpoint I see no reason to take it this far. You could still proxy the OS "long press gestures" to right clicks for the time being, but the OS would be responsible for actually detecting them.

Let me clarify, once the long press gesture is triggered, the held touch will have to be ignored, and not cause a left mouse click (otherwise you could enter beatmaps when wanting to right-click on them in the carousel). This is not feasible without introducing a “cancelled” state to all mouse button input which will be initiated from touch input handlers, and that may require a good amount of effort, as opposed to the current logic in this PR.

I’m willing to give it a try on iOS and see where it leads.

[bdach]

Yeah I don't know. I'm uneasy with rolling our own code for press vs long press detection if the OSes already have got it handled somehow. "Cancelled" mouse inputs, or the IgnoreClick you have in this diff, don't exactly inspire confidence in me. But maybe that's just me.

[frenzibyte]

Also realized this will break any component which requires the user to hold onto (mainly osu! ruleset, holding slider)... Will need to rethink the mouse-from-touch flow a bit, in addition to trying out OS gestures.

[frenzibyte]

Upon trying UILongPressGestureRecognizer, the touch will be cancelled after the gesture is triggered, as expected. This will break UI components that require more than 0.5s of holding (dangerous buttons for example).

This is at least salvageable with the right click logic being at InputManager, by keeping the touch in a held state until it's released, and release it with IgnoreClick = true.

[frenzibyte]

The following changes should fix the concern mentioned above regarding left click releasing before right click, and shows that it's not quite feasible relying on OS gestures with that behaviour.

There are still other points I'm not sure about as they may add more complexity, such as:

• Should right click be activated when holding with one finger than another and releasing the first?
• Should right click be activated when holding with one finger than another and releasing the other?
• Should right click be activated when holding with one finger and swiping with it before it's triggered? Or should it reset the counter every touch move? Or should it never activate.

I would lean towards no for all, but I'm not sure if it's worth the complexity. The last part sounds like a must as it would break scrolling UX, will fix that.

[peppy]

Should only be handled on a single touch, if another finger touches it should permanently cancel until all are released. Non-transferrable between fingers. aka, keep it simple.

• Should right click be activated when holding with one finger and swiping with it before it's triggered? Or should it reset the counter every touch move? Or should it never activate.

Moving the touch beyond a certain amount should cancel it. Probably the same as the drag lenience.

[frenzibyte]

Have addressed all of the points above with a bit of a refactor, this feels quite good on thorough testing for me now. Will look into visual feedback in a bit.

[frenzibyte]

Have added support for visual feedback with pretty basic/wanky radial design as an initial step. Can probably be further improved.

[peppy]

Regarding the conversation of using the operating system level gesture / long touch flow: I do agree it is something we should eventually aim for, but it comes with some hard issues that may take a paradigm shift to fix (making our input flow match closer to mobile expectations).

The way it's done in this PR seems like a good initial step which will at least give us the behaviour we want in a generically accepted form. Can iterate at a later point based on user feedback.

[peppy]

I finally got around to testing this out on some devices. Sorry for the delay.

I've made some visual tweaks as I couldn't see the circle on smaller devices (like the iPhone SE 4.5" or whatever it was).

Remaining issues as I see:

• After starting a drag (ie. cancelling the hold), a new hold should not start until the finger has been lifted. Right now it starts again whenever the input is still
• The leniency seems a bit too low, it's easy to accidentally move the input too much while trying to hold

Both of these issues can be seen in this capture, where I do not lift my finger once:

https://user-images.githubusercontent.com/191335/194473455-507ea9ec-a78e-4384-b3f2-0dae7ac13485.mov

[frenzibyte]

After starting a drag (ie. cancelling the hold), a new hold should not start until the finger has been lifted. Right now it starts again whenever the input is still

This wouldn't be the case when the drag is handled by a drawable. That is the simplest way to go about it, but given that leniency is required as well. Will have it work by checking against touch down position instead.

[frenzibyte]

The mentioned issues should be resolved now, haven't managed to test these changes on a physical device due to latest Xcode version being a nuisance (need to update xamarin packages, but internet speed sucks). @peppy would appreciate checking the latest changes and see if it feels better now.

[peppy]

Now the position is locked, but it can still be triggered multiple times... and if you move a touch back to the original location it starts triggering again, even though you've already moved away (should not happen).

[Ghost1928563945]

This could be a different event too. Maybe like the ios force touch has a different design and better user interaction feedback.'

[frenzibyte]

Now the position is locked, but it can still be triggered multiple times... and if you move a touch back to the original location it starts triggering again, even though you've already moved away (should not happen).

Spent time to rewrite the entire logic to work with touch dragging properly, as the previous one wasn't quite cut for it. Should work correctly now.

[peppy]

I'll give this some more testing on a device, but the new test coverage looks ample to cover the cases I pointed out 👍 .

[dengr1065]

Is it planned to make long-press duration follow the accessibility system setting on Android? Looks like you can access it without receiving long-press gesture events: ViewConfiguration#getLongPressTimeout