Adding Apple Pencil Touch to Konva in Label Studio

[madhavajay]

trafficstars

Hi, I am trying to add support for the apple pencil (using touch) in Brush annotation mode on the awesome Label Studio which uses Konva. https://github.com/heartexlabs/label-studio/issues/1982

I have debugged it using the frontend tooling they provide: https://github.com/heartexlabs/label-studio-frontend

What I have discovered is that the normal ipad mouse (keyboard and track pad) does all the normal touch events fine, but when the touch or pencil triggers it seems to hit a preventDefault / stopPropagation type event.

I saw mention in another issue that you can add events to the Stage element, however when I added some touch events that passed through the same functionality as mouse it doesn't work.

I have also toyed with putting style="pointer-events: none; touch-action: none" on a higher div to prevent the internal scroll overflow from capturing the touch focus and then adding something like pointer events all or touch action auto on the div above the canvas, however I think I still mis-understand something about how this works.

Do you have any ideas or suggestions on how I can suppress the x and y scroll on nested divs above the Konva Canvas and also enable the touch / pencil events to mirror the same as whats happening with the normal mouse?

I am both new to Konva and not very experienced with React or the Label Studio Frontend so im still trying to figure out which part is responsible for the issue, but it feels like its a combination:

• touch events being captured by scrolling elements and page size too large
• konva preventing the touch from registering
• potentially some difference between touch and mouse which means I can't just map them 1:1

[lavrton]

I usually just map all touch events into the same mouse handlers. E.g.:

<Stage
 onMouseDown={handleMouseDown}
 onTouchStart={handleMouseDown}
/>

And usually, it works good for me. Also, you can try to use pointer events.

[madhavajay]

@lavrton sorry for the slow reply, I finally got to try, looks like I needed to use .layerX and .layerY as well and some other stuff: https://github.com/heartexlabs/label-studio-frontend/pull/802

I guess that might be implementation details from label-studio-frontend rather than konvajs, although I would have imagined it would be nice if something like konva just handled touch vs mouse events internally.