🐛 Frame data is wrong on special device

[vanhungoz]

What were you trying to do?

i want to use frame processor

Reproduceable Code

const frameProcessor = useFrameProcessor((frame) => {
    "worklet";
   console.log(frame)
  }, []);

What happened instead?

i save frame to file, and it is how it look like

Relevant log output

No response

Device

vsmart joy 4

VisionCamera Version

"react-native-vision-camera": "^2.12.2"

Additional information

• [ ] I am using Expo
• [X] I have read the Troubleshooting Guide
• [X] I agree to follow this project's Code of Conduct
• [X] I searched for similar issues in this repository and found none.

[mrousavy]

Incomplete issue. There is literally nothing for me to reproduce, and I have no idea what you're doing.

i save frame to file, and it is how it look like

this does not tell me anything. I cannot reproduce "save frame to file", no code was shared.

[vanhungoz]

fun Image.toBitmap(): Bitmap { val yBuffer = planes[0].buffer // Y val vuBuffer = planes[2].buffer // VU val ySize = yBuffer.remaining() val vuSize = vuBuffer.remaining() val nv21 = ByteArray(ySize + vuSize) yBuffer.get(nv21, 0, ySize) vuBuffer.get(nv21, ySize, vuSize) val yuvImage = YuvImage(nv21, ImageFormat.NV21, this.width, this.height, null) val out = ByteArrayOutputStream() yuvImage.compressToJpeg(Rect(0, 0, yuvImage.width, yuvImage.height), 100, out) val imageBytes = out.toByteArray() return BitmapFactory.decodeByteArray(imageBytes, 0, imageBytes.size) }

private fun saveBitmapToFile(bit: Bitmap){ try { val fileName = Date().time val fileOutputStream: FileOutputStream = this.context.openFileOutput(fileName.toString()+".jpg", Context.MODE_PRIVATE) bit.compress(Bitmap.CompressFormat.JPEG, 100, fileOutputStream) fileOutputStream.close() } catch (e: java.lang.Exception) { e.printStackTrace() Log.e("error",e.message.toString()) } }

override fun callback(image: ImageProxy, params: Array<Any>): Any? { Log.e("rotationDegrees", image.imageInfo.rotationDegrees.toString()) try { @SuppressLint("UnsafeOptInUsageError") val bit = image.image!!.toBitmap() saveBitmapToFile(bit)

[vanhungoz]

@mrousavy
i'm sorry for missing information it only happen on vsmart joy 4

[vanhungoz]

other devices is still working correctly

[vanhungoz]

@mrousavy could you reopen it?

[mrousavy]

hm, are we sure that this is a YUV buffer?

[mrousavy]

Try this one:

private Bitmap toBitmap(Image image) {
    Image.Plane[] planes = image.getPlanes();
    ByteBuffer yBuffer = planes[0].getBuffer();
    ByteBuffer uBuffer = planes[1].getBuffer();
    ByteBuffer vBuffer = planes[2].getBuffer();

    int ySize = yBuffer.remaining();
    int uSize = uBuffer.remaining();
    int vSize = vBuffer.remaining();

    byte[] nv21 = new byte[ySize + uSize + vSize];
    //U and V are swapped
    yBuffer.get(nv21, 0, ySize);
    vBuffer.get(nv21, ySize, vSize);
    uBuffer.get(nv21, ySize + vSize, uSize);

    YuvImage yuvImage = new YuvImage(nv21, ImageFormat.NV21, image.getWidth(), image.getHeight(), null);
    ByteArrayOutputStream out = new ByteArrayOutputStream();
    yuvImage.compressToJpeg(new Rect(0, 0, yuvImage.getWidth(), yuvImage.getHeight()), 75, out);

    byte[] imageBytes = out.toByteArray();
    return BitmapFactory.decodeByteArray(imageBytes, 0, imageBytes.length);
}

[vanhungoz]

i'm sorry for my late response after few days of trying, i figure out that: i only work when i comment all function imageAnalysisBuilder.setTargetAspectRatio(aspectRatio) (if no format is set) and imageAnalysisBuilder.setTargetResolution(format.videoSize) (if format is set)

and it only work if i set setTargetResolution(480, 640) at the line https://github.com/mrousavy/react-native-vision-camera/blob/65d4d46f6ab4e04dca15c15b8cf9c07a17f97d72/android/src/main/java/com/mrousavy/camera/CameraView.kt#L396

other value is not work. Do you have any suggestion to figure out this?

[vokhuyetOz]

const item = cameraDevice.formats.find((item) => { return ( item.videoWidth > 600 && item.videoWidth < 700 && item.videoHeight > 400 && item.videoHeight < 500 ); }); setCameraConfig({ preset: undefined, format: { ...item, videoWidth: 480, videoHeight: 640 }, });

my workaround is fixed resolution for only this device model.

[vanhungoz]

it only appear on special device. so i close it now

[bekatd]

I am getting same issue with Xiaomi mi A2 lite and also was getting same issue with emulator too (but not always)

[bekatd]

this issue should be open. If I use workaround suggested above, than I sacrifice other devices, which do not support suggested resolution :|

[mrousavy]

This is now fixed in V3! 🥳