label-studio-frontend
label-studio-frontend copied to clipboard
fix: iOS touch in segmentation brush mode
This PR attempts to fix the touch input issues on label studio when using iPad Apple Pencil and brush / segmentation mode.
It isn't 100% perfect but ImageView.js is complex and I don't pretend to know all the edge cases.
There are some videos here showing the issue and it working afterwards. https://imgur.com/a/fY4LsSf
It seems like @lavrton suggestion of mapping the events directly isnt enough: https://github.com/konvajs/konva/issues/1361#issuecomment-1181918694
From what I can tell, the MouseEvent's have .offsetX and .offsetY however the TouchEvent's don't have these, but do have some similar values such as .layerX and .layerY. 🤷♀️.
Additionally, there is the need to pinch and zoom, but without e.preventDefault() doing things is particularly challenging.
My guess is that something like this would be optimal in the long term: https://github.com/shuding/apple-pencil-safari-api-test
I know theres a lot of people using computers to label, but honestly, for brush style annotations, surely iPad support should be a #1 priority.
I am happy to help get this ready for merge if someone can help me understand what would be next to do.
Thank you for your amazing contribution!
@hogepodge please take a look at this PR
/git merge master
Error: Merge conflict detected, please resolve it using the git command line.
/git merge master
Error: Merge conflict detected, please resolve it using the git command line.
This PR attempts to fix the touch input issues on label studio when using iPad Apple Pencil and brush / segmentation mode.
It isn't 100% perfect but
ImageView.jsis complex and I don't pretend to know all the edge cases.There are some videos here showing the issue and it working afterwards. https://imgur.com/a/fY4LsSf
It seems like @lavrton suggestion of mapping the events directly isnt enough: konvajs/konva#1361 (comment)
From what I can tell, the
MouseEvent's have.offsetXand.offsetYhowever theTouchEvent's don't have these, but do have some similar values such as.layerXand.layerY. 🤷♀️.Additionally, there is the need to pinch and zoom, but without
e.preventDefault()doing things is particularly challenging.My guess is that something like this would be optimal in the long term: https://github.com/shuding/apple-pencil-safari-api-test
I know theres a lot of people using computers to label, but honestly, for brush style annotations, surely
iPadsupport should be a #1 priority.I am happy to help get this ready for merge if someone can help me understand what would be next to do.
sadly,when the image become large,like 1M+, the touch draw region saved nothing after submit or update