label-studio-frontend icon indicating copy to clipboard operation
label-studio-frontend copied to clipboard

fix: iOS touch in segmentation brush mode

Open madhavajay opened this issue 3 years ago • 5 comments

This PR attempts to fix the touch input issues on label studio when using iPad Apple Pencil and brush / segmentation mode.

It isn't 100% perfect but ImageView.js is complex and I don't pretend to know all the edge cases.

There are some videos here showing the issue and it working afterwards. https://imgur.com/a/fY4LsSf

It seems like @lavrton suggestion of mapping the events directly isnt enough: https://github.com/konvajs/konva/issues/1361#issuecomment-1181918694

From what I can tell, the MouseEvent's have .offsetX and .offsetY however the TouchEvent's don't have these, but do have some similar values such as .layerX and .layerY. 🤷‍♀️.

Additionally, there is the need to pinch and zoom, but without e.preventDefault() doing things is particularly challenging.

My guess is that something like this would be optimal in the long term: https://github.com/shuding/apple-pencil-safari-api-test

I know theres a lot of people using computers to label, but honestly, for brush style annotations, surely iPad support should be a #1 priority.

I am happy to help get this ready for merge if someone can help me understand what would be next to do.

madhavajay avatar Aug 06 '22 04:08 madhavajay

Thank you for your amazing contribution!

makseq avatar Aug 09 '22 00:08 makseq

@hogepodge please take a look at this PR

makseq avatar Feb 28 '23 17:02 makseq

/git merge master

Error: Merge conflict detected, please resolve it using the git command line.

Workflow run

hlomzik avatar Oct 02 '23 15:10 hlomzik

/git merge master

Error: Merge conflict detected, please resolve it using the git command line.

Workflow run

farioas avatar Oct 10 '23 15:10 farioas

This PR attempts to fix the touch input issues on label studio when using iPad Apple Pencil and brush / segmentation mode.

It isn't 100% perfect but ImageView.js is complex and I don't pretend to know all the edge cases.

There are some videos here showing the issue and it working afterwards. https://imgur.com/a/fY4LsSf

It seems like @lavrton suggestion of mapping the events directly isnt enough: konvajs/konva#1361 (comment)

From what I can tell, the MouseEvent's have .offsetX and .offsetY however the TouchEvent's don't have these, but do have some similar values such as .layerX and .layerY. 🤷‍♀️.

Additionally, there is the need to pinch and zoom, but without e.preventDefault() doing things is particularly challenging.

My guess is that something like this would be optimal in the long term: https://github.com/shuding/apple-pencil-safari-api-test

I know theres a lot of people using computers to label, but honestly, for brush style annotations, surely iPad support should be a #1 priority.

I am happy to help get this ready for merge if someone can help me understand what would be next to do.

sadly,when the image become large,like 1M+, the touch draw region saved nothing after submit or update

lackray avatar Jan 03 '24 08:01 lackray