winit icon indicating copy to clipboard operation
winit copied to clipboard
verified publisher icon rust-windowing

Adding touchpad multitouch support

Open simon-frankau opened this issue 3 years ago • 4 comments

Hi.

I'm interested in adding touchpad multitouch support to winit, with an implementation for MacOS. I'm new to winit, so I'd be keen to get feedback on whether this is worth implementing before I start coding. I'll try to pre-empt a few questions here:

  • What is "touchpad multitouch support"? winit already supports multitouch, but (I believe) it assumes that you're touching the screen, so touches are in screen coordinates. This would be supporting touches where the coordinates are touchpad coordinates.
  • Why would this support be useful? I'm aware of #2157 - touchpad gestures. This is another approach: Having the raw touches available allows extra flexibility, so that the library user can interpret a wider range of multitouch movements. For example, the touches can be interpreted to provide pinch-to-zoom with different scales on the different axes, or custom gestures.
  • So, couldn't similar functionality be provided by extending #2157?. Yes-ish. Recognition for extra gestures could be coded into winit, but passing touches straight through to the user seems more flexible and potentially less complex for winit, if you were to support such gestures.
  • Is this instead of #2157? No, this provides more flexibility for those that want it, enabling access to raw touches, as well as gestures. Gestures are still very useful.
  • Should winit supporting touchpad multitouch? Looking at the list in FEATURES.md, there is already support for a) multitouch and b) touchpad functionality. It feels to me like it's pretty much inside the boundaries defined by existing features, although I do have sympathy for "Yeah, but can we avoid scope creep on potentially marginal features?".
  • What would this look like, API-wise? I think the assumption is that Touch coordinates are screen coordinates. My suggestion would be to add an enum member to Touch for the coordinate system used - something like "screen" for touch screen, and "device" for trackpad. Maybe these events should be delivered as DeviceEvents rather than WindowEvents?
  • Why do you personally want this? While I hope this feature would be useful to other people, I'd like egui to support zooming by different amounts in the X and Y axes from the same gesture, which is not supported with the MacOS pinch gesture, but is supported with multitouch.
  • What platforms would you support? I only have MacOS to hand, so my implementation would only cover MacOS.

As I said at the start, I'm new to winit, so I may have made some incorrect assumptions, or be thinking about this incorrectly. Please do correct me.

Thanks, Simon.

simon-frankau avatar Jul 18 '22 23:07 simon-frankau

I suppose that would be useful, but I'm not sure if we can implement it on other platforms (or how you'd even implement it on macOS).

maroider avatar Jul 20 '22 09:07 maroider

We'll probably be merging https://github.com/rust-windowing/winit/pull/2157, so that part is at least covered.

Good issue description, I agree that this is in-scope and the path you've laid out (modifying the Touch enum) sounds fine. Ping me when you have something!

madsmtm avatar Aug 10 '22 17:08 madsmtm

Ooops, sorry for not replying earlier - some "real-world issues" got in the way.

If "single platform only implementation" is an problem, I can understand if people want to call it out of scope. I can also take a quick look to see if other platforms have a way to support this (even if I'm not in a position to implement it myself).

My proof-of-concept implementation relies on listening for touches{Began,Moved,Ended,Cancelled}WithEvent: events. The main issues I have so far are:

  • Lack of familiarity with the Apple APIs.
  • Thoughts on how to correctly handle "multi-touch" - I think this API also reports single touch events as a sub-set of multi-touch events, and probably these should be filtered out given they're also reported as mouse movements. It feels like there are corner cases worth getting right.

I'll try to follow up when I have something worth looking at, but the "real-world issues" mentioned above might still slow me down somewhat. In the meantime, I'm glad to hear #2157 is likely to get merged.

simon-frankau avatar Aug 13 '22 16:08 simon-frankau

Take your time, no worries!

Thoughts on how to correctly handle "multi-touch"

Honestly, no idea, but I guess any API is better than no API, so just getting something that somewhat works to begin with would be an improvement

madsmtm avatar Aug 14 '22 00:08 madsmtm