blockly icon indicating copy to clipboard operation
blockly copied to clipboard

Ignore mousedown for multiple touch points

Open breville opened this issue 5 years ago • 2 comments

This is a proposal to only handle mousedown (well, really touchstart) events when a single touch point is involved. But I'm not sure whether it's actually a good idea or not, so am using this PR to capture what I've learned in the last day or so.

We have a challenge viewing our level pages on mobile devices, related to zoom. Especially when switching between portrait and landscape, users can end up zoomed in, and need to zoom back out to see the whole screen. Unfortunately, the Blockly area "takes over" the ability to zoom and pan on a mobile phone, and so the user has to find somewhere on the page to grab that actually lets the browser do its zoom and pan.

Blockly "takes over" through a preventDefault on mousedown here. Commenting out that line allows the user to use one finger to scroll the whole page, and two fingers to zoom. However, Blockly does this takeover because has a custom scrolling implementation, and it allows a mouse drag or touch drag to control the scrolling anywhere on its pane. The latest Blockly demo also exhibits the same behaviour on a mobile phone.

I had hoped that we might be able to detect any touch with more than one finger, and then pass control back to the browser, but it appears that any touchstart that encounters a preventDefault will take over control from the browser. This means that if one finger is placed first, and then a second goes down, the browser has already lost the ability to own the zoom. (I'd love to be found wrong on this!)

And so all this PR manages to offer is detecting a touchstart when two fingers are placed simultaneously. In this case, Blockly does avoid taking over, and allows the user to then do their zoom. It's pretty tricky to do this gesture, though, and the added risk is that they could do it once to make their zoom worse, and then not be able to do it again to make it better.

The better long term solution for us it to probably prevent Blockly from doing this takeover, either by

  • always having a pane big enough for the entire program, or
  • implementing a browser-native scrolling surface inside Blockly

The former becomes a factor as we think about the right mobile experience for an entire level page; it might still be useful even if we want to show a pane bigger than a mobile device's physical screen. The latter still has complications in any future native experience we build, because scrolling components inside a scrolling page have their own difficulties, especially on mobile devices.

breville avatar Apr 10 '20 21:04 breville

This means that if one finger is placed first, and then a second goes down, the browser has already lost the ability to own the zoom. (I'd love to be found wrong on this!)

Do you know if there's anything detectable that happens when that second finger goes down? Is a second touchstart fired, or do the following touchmoves look different?

Hamms avatar Apr 13 '20 18:04 Hamms

@Hamms Good questions. I took some notes on what seems to happen:

First finger down
touchstart     event.touches.length: 1     event.changedTouches.length: 1

Second finger down
touchstart     event.touches.length: 2     event.changedTouches.length: 1

Move one finger around
touchmove      event.touches.length: 2     event.changedTouches.length: 1

or

Two fingers down at same time
touchstart     event.touches.length: 2     event.changedTouches.length: 2

It really seems to be that once we preventDefault on the first finger down, we take away the ability of the browser to manage a one-finger pan or a two-finger pinch-zoom after that, even if we don't preventDefault on the second finger down.

I guess this explains the UI in embedded Google Maps on mobile devices in which one finger on the map goes through to the browser so that you can scroll the page, while it shows a message that two fingers should be used to control the map, and then two fingers indeed control the map's own panning and zooming. We could consider this for our Blockly embeds on touch devices.

breville avatar Apr 14 '20 08:04 breville