Android: Controls Haptic Feedback

[codokie]

This PR adds haptic feedback support for input overlay touch controls ~~as well as sliders, button toggles and more~~ will be done in #12984.

There is a new dialog in "Overlay Controls" -> "Touch Haptics" for enabling any of the following haptic feedback triggers (they are all disabled by default):

1. Overlay button press -> Quick fall effect with a fallback to a vibration of 100ms and 70% amplitude at most
2. Overlay button release -> Quick rise effect with a fallback to a vibration of 70ms and 50% amplitude at most
3. Overlay joystick movement -> Low tick effect with a fallback to a vibration of 50ms and 35% amplitude at most

The maximum recommended duration is 20 ms for a short vibration effect according to the Android docs, but some low-end or unoptimized vibrators may not vibrate at all if the duration is that low.

You may also notice that the controller vibration has been replaced with a (rich haptics) spin effect. I've moved the content of vibration related functions from ControllerInterface.kt to a new object so I could use them in the new HapticsProvider class without duplicating code.

In the dialog mentioned before there is a slider for adjusting the intensity (duration and amplitude) of the vibration. It applies only to the input overlay haptics and does not affect the intensity of the controller vibration which is issued to the vibration motor selected in the Rumble options. It would have been ideal if there was an additional intensity slider under the Rumble options, but it seems that the controller settings are retrieved from the C++ core and I am not sure how a new Android-specific setting should be added there.

Please let me know what you think or if you have found an issue. Thanks

[codokie]

Okay so now in Android 12+ the vibrations are composed with primitives if the device supports all 4 effects: ~~CLICK, TICK,~~ QUICK_RISE, QUICK_FALL, SPIN and LOW_TICK. Not sure how many devices actually support them all, but I know that the Galaxy S23 does, and I suppose other (recent) flagship phones do as well.

I've replaced the duration & amplitude sliders with a single ~~5~~ 10 steps "intensity" slider as it's more intuitive this way, and because you can only specify a scale for a primitive composition.

ControllerInterface.vibrate() will use the SPIN effect for rich haptics at medium intensity (not scalable due to a lack of a slider in the Rumble options). It should be a bit weaker and shorter overall than the old vibration, which was pretty buzzy.

Not sure if it affects controllers with special support in Dolphin like a real Wii Remote/GameCube controller (do they use the ControllerInterface for rumble?)

On a side note: changing the intensity will now adjust the sensitivity of the input overlay joystick.

[JosJuice]

This seems like a useful feature, making it easier to tell whether your finger properly hit the touch button or not. But I'll have to ask someone else to test rumble with controllers, as I don't have any compatible controller with rumble.

There's currently a lot of commits in this PR. Could you rebase the branch to make the history cleaner? We generally don't want any commits that fix things that were wrong in earlier commits in the same PR.

Not sure if it affects controllers with special support in Dolphin like a real Wii Remote/GameCube controller (do they use the ControllerInterface for rumble?)

No, they don't. Only controllers that you can set up button mappings for use ControllerInterface. For Wii Remotes and GameCube controllers, Dolphin sends rumble signals in the same way a real console would, both in master and with this PR.

[codokie]

This seems like a useful feature, making it easier to tell whether your finger properly hit the touch button or not

Indeed, it would help those who don't have a controller or prefer touch controls. It is especially useful for rhythm games and other games which require precise/well-timed input.

Could you rebase the branch to make the history cleaner?

Sure sorry about the mess, I did not know I could clean the history of pushed commits. ~~It seems 3277b8b and all commits prior it should be squashed to get rid of the code that moved to #12984. The commits following it seem like fixup commits, does that sound right?~~ Edit: I rebased the 14 commits, now there are only 5. I hope this is clean enough, if not I will squash some more. The date of the commits has been affected for some reason, not sure I could have avoided that.

No, they don't. Only controllers that you can set up button mappings for use ControllerInterface.

Thanks for the clarification, so controllers with special support like Wiimote do not need to be tested then. Opinions on the vibration intensity of regular controllers would still be valuable as long as there is no slider for adjusting it in the Rumble options.

[JosJuice]

There's still quite a bit of "code motion" where you do large overhauls to code introduced in previous commits. It would be better if it was squashed more, maybe even into one single commit.

As for the dates, it looks to me like the author date is remains an earlier date, but the commit date was updated. (GitHub only shows the commit date.) That's perfectly normal. You can forge the commit date if you really want to, but there's no reason to do so.

[codokie]

No problem, I can squash them all. The date of the commit does not bother me. The only thing missing right now is a slider for the controller rumble in the Rumble options. Any idea how to add one only for Android devices? I think it may be done in GCPadEmu.cpp and WiimoteEmu.cpp There should also be a get function to retrieve the value of this slider in ControllerInterface

[JosJuice]

We actually already have a setting for this in Dolphin, it just isn't exposed on Android. It's called "Multiplier" in the GUI and "range" in the source code. If my understanding is correct, the value of that setting will be reflected in the state variable here: https://github.com/dolphin-emu/dolphin/blob/d4ec524f218f83bf3646f0dd666236307a727bc5/Source/Core/InputCommon/ControllerInterface/Android/Android.cpp#L574

So the work needed is:

1. Add the Multiplier setting to the Android GUI
2. Pass the state value into the Kotlin code instead of just checking if it's above 0.5

I can work on number one, but I'm not sure how soon I'll have time for it. (The steps needed are a bit complicated to explain, which is why I'm suggesting that I would do it.) Do you feel this is important to include in this PR, or would it be fine to move on with getting this PR merged for now and then adding the setting?

Also, just as a note: Games can trigger rumble for different lengths of time. Currently we treat all lengths of time as the same on Android, since the code is only checking for the rising edge of the rumble signal. Not sure if Android offers any convenient way to have rumble continue for an amount of time that we don't know upfront.

[codokie]

That's cool I did not know there was a multiplier setting for the controller vibration already available in the core. I would be glad to help with Java/Kotlin code should you need any assistance, sadly I'm not too familiar with C++.

Do you feel this is important to include in this PR, or would it be fine to move on with getting this PR merged for now and then adding the setting?

It isn't mandatory and may be added later because even without this PR, there is no way to adjust the intensity of the controller rumble. From my testings, the new vibration should be less intense than the old one but not too faint.

Not sure if Android offers any convenient way to have rumble continue for an amount of time that we don't know upfront.

Passing 0 as the 'repeat' argument to createWaveform will repeat the vibration indefinitely until vibrator.cancel() is issued. I'm afraid the API does not have an equivalence for composition of PRIMITIVE_SPIN, but the controller rumble doesn’t have to be a ‘spin’ effect if that would mean the duration isn’t variable or faithful to the source. If there is a way to estimate the duration of each vibration during runtime, then you could scale the timings array passed to createWaveform or compose a vibration of multiple primitives.

[codokie]

Closing this in favor of #13220 w/o the controller rumble changes which still needs to be worked on.