vision-camera-image-labeler
vision-camera-image-labeler copied to clipboard
mrousavy
Image labelling not working on iOS
So I tested example app on iOS 15 iPhone 12 and it's not working properly. Results are always same:
- sky
- pattern
- curtain
Results have nothing to do with what's in the camera frame.
From logs I get this in xcode:
[WARNING]The specified colorspace format is not supported. Falling back on Libyuv.
I also tested the example app and image labelling works in iPhone 6 and iPhone 8, but in the bigger version labelling does not work only in portrait mode ( always returning the same result ), in landscape orientation working fine.
One thing I figured out is that if you set preset="cif-352x288" for <Camera /> it gives better results but I think it should work with better quality 🤔 will continue to debug.
I think this is because iOS has new 10-bit x420 formats that are not supported by the image labeller. Maybe we can upgrade the native library to support it, or choose a format that's not x420. Thomas is working on a PR in VisionCamera to expose that prop in format.
Yeah, I came to similar conclusion. Here's the PR @mrousavy mentioned: https://github.com/mrousavy/react-native-vision-camera/pull/559 and it will hopefully solve this. My hunch is that MLKit is now receiving "images" something like in this Stack Overflow post https://stackoverflow.com/questions/42997462/convert-cmsamplebuffer-to-uiimage and that's why it is always returning pattern, curtain, sky, monochrome results. 🤷♂️
I'm still receiving this issue. I upgraded to RNVC 2.12.0, which should include mrousavy/react-native-vision-camera#559. My pixelFormat is set to 420v, but the issue remains.
Any ideas @valstu?
