react-native-vision-camera
react-native-vision-camera copied to clipboard
mrousavy
🐛 Frame processor + take photo causes crashes
What's happening?
Calling camera.takePhoto after switching the device causes crashes when the frame processor is performing reads on the buffer. This behavior was noticed within my application, which involves performing ML tasks on frames.
If you run the null frame processor which does nothing within it, camera.takePhoto works well.
Also, running the example frame processor plugin doesn't cause crashes.
- Run the example app with the following frame processor code
- Switch the device from back to front
- Call
takePhoto
Reproduceable Code
const frameProcessor = useFrameProcessor((frame) => {
'worklet'
// Commenting this out makes crashes not happen
frame.toArrayBuffer()
}, [])
Relevant log output
2023-11-08 20:24:16.648 31765-31830 ReactNativeJS com.mrousavy.camera.example I Camera: FRONT (1) | Format: (2400x1080 photo / 1920x864@60 video @ 60fps)
2023-11-08 20:24:16.659 31765-31765 PreviewView com.mrousavy.camera.example I Resizing PreviewView to 1920 x 864...
2023-11-08 20:24:16.660 31765-31765 CameraView com.mrousavy.camera.example I Updating CameraSession...
2023-11-08 20:24:16.660 31765-31860 CameraSession com.mrousavy.camera.example I Updating CameraSession Configuration...
2023-11-08 20:24:16.661 31765-31860 CameraSession com.mrousavy.camera.example I Configuring Camera #1...
2023-11-08 20:24:16.745 31765-32005 CameraManagerGlobal com.mrousavy.camera.example I Camera 0 facing CAMERA_FACING_BACK state now CAMERA_STATE_IDLE for client com.mrousavy.camera.example API Level 2User Id 0
2023-11-08 20:24:16.860 31765-32005 CameraManagerGlobal com.mrousavy.camera.example I postSingleUpdate device: camera id 0 status STATUS_PRESENT
2023-11-08 20:24:16.860 31765-31831 CameraDevices com.mrousavy.camera.example I Camera #0: Available!
2023-11-08 20:24:16.860 31765-32005 CameraManagerGlobal com.mrousavy.camera.example I Camera 0 facing CAMERA_FACING_BACK state now CAMERA_STATE_CLOSED for client com.mrousavy.camera.example API Level 2User Id 0
2023-11-08 20:24:16.861 31765-31778 .camera.example com.mrousavy.camera.example W Long monitor contention with owner mrousavy/VisionCamera.main (31860) at void android.hardware.camera2.impl.CameraDeviceImpl.close()(CameraDeviceImpl.java:1463) waiters=0 in void android.hardware.camera2.impl.CameraDeviceImpl.onDeviceError(int, android.hardware.camera2.impl.CaptureResultExtras) for 124ms
2023-11-08 20:24:16.861 31765-31860 CameraManager com.mrousavy.camera.example I Camera 1: Opening...
2023-11-08 20:24:16.875 31765-32122 CameraManagerGlobal com.mrousavy.camera.example I postSingleUpdate device: camera id 1 status STATUS_NOT_AVAILABLE
2023-11-08 20:24:16.875 31765-31831 CameraDevices com.mrousavy.camera.example I Camera #1: Unavailable!
2023-11-08 20:24:16.883 31765-32122 CameraManagerGlobal com.mrousavy.camera.example I Camera 1 facing CAMERA_FACING_FRONT state now CAMERA_STATE_OPEN for client com.mrousavy.camera.example API Level 2User Id 0
2023-11-08 20:24:16.883 31765-31860 CameraManager com.mrousavy.camera.example I Camera 1: Opened!
2023-11-08 20:24:16.887 31765-31860 PreviewView com.mrousavy.camera.example I Resizing PreviewView to 1920 x 864...
2023-11-08 20:24:16.887 31765-31860 CameraSession com.mrousavy.camera.example I Successfully configured Camera #1!
2023-11-08 20:24:16.891 31765-31860 CameraSession com.mrousavy.camera.example I Configuring Session for Camera #1...
2023-11-08 20:24:16.892 31765-31860 SurfaceOutput com.mrousavy.camera.example I Closing 2400x1080 PHOTO ImageReader..
2023-11-08 20:24:16.892 31765-31860 SurfaceOutput com.mrousavy.camera.example I Closing 2400x1080 Video Pipeline..
2023-11-08 20:24:16.893 31765-31860 CameraSession com.mrousavy.camera.example I Adding 2400 x 1080 Photo Output in Format #256...
2023-11-08 20:24:16.894 31765-31860 CameraSession com.mrousavy.camera.example I Adding 1920 x 864 Video Output in Format #34...
2023-11-08 20:24:16.894 31765-31860 VideoPipeline com.mrousavy.camera.example I Initializing 1920 x 864 Video Pipeline (format: NATIVE)
2023-11-08 20:24:16.894 31765-31860 VideoPipeline com.mrousavy.camera.example I Using ImageReader round-trip (format: #34)
2023-11-08 20:24:16.894 31765-31860 VideoPipeline com.mrousavy.camera.example I Using API 29 for GPU ImageReader...
2023-11-08 20:24:16.895 31765-31860 CameraSession com.mrousavy.camera.example I Adding 1920 x 864 Preview Output...
2023-11-08 20:24:16.895 31765-31860 PreviewView com.mrousavy.camera.example I Resizing PreviewView to 1920 x 864...
2023-11-08 20:24:16.898 31765-31860 CreateCaptureSession com.mrousavy.camera.example I Camera 1: Creating Capture Session #1002... Hardware Level: 0} | Outputs: [android.hardware.camera2.params.OutputConfiguration@70575b5d, android.hardware.camera2.params.OutputConfiguration@fc2c2196, android.hardware.camera2.params.OutputConfiguration@8def04b6]
2023-11-08 20:24:16.898 31765-31860 CreateCaptureSession com.mrousavy.camera.example I Using new API (>=28)
2023-11-08 20:24:16.994 31765-31860 CreateCaptureSession com.mrousavy.camera.example I Camera 0: Capture Session #1001 closed!
2023-11-08 20:24:16.994 31765-31860 CreateCaptureSession com.mrousavy.camera.example I Camera 1: Capture Session #1002 configured!
2023-11-08 20:24:16.994 31765-31860 CameraSession com.mrousavy.camera.example I Successfully configured Session with 3 outputs for Camera #1!
2023-11-08 20:24:16.994 31765-31860 CameraSession com.mrousavy.camera.example I Updating Video Outputs...
2023-11-08 20:24:16.994 31765-31860 VideoPipeline com.mrousavy.camera.example I Setting 1920 x 864 FrameProcessor Output...
2023-11-08 20:24:16.994 31765-31860 VideoPipeline com.mrousavy.camera.example I Removing RecordingSession Output...
2023-11-08 20:24:16.996 31765-31860 CameraSession com.mrousavy.camera.example I Successfully updated CameraSession Configuration! isActive: true
2023-11-08 20:24:16.996 31765-31860 CameraView com.mrousavy.camera.example I invokeOnInitialized()
2023-11-08 20:24:16.997 31765-32122 CameraManagerGlobal com.mrousavy.camera.example I Camera 1 facing CAMERA_FACING_FRONT state now CAMERA_STATE_ACTIVE for client com.mrousavy.camera.example API Level 2User Id 0
2023-11-08 20:24:16.997 31765-31830 ReactNativeJS com.mrousavy.camera.example I Camera initialized!
2023-11-08 20:24:17.323 31765-31861 VideoPipeline com.mrousavy.camera.example I ImageReader::onImageAvailable!
2023-11-08 20:24:17.324 31765-31861 Frame com.mrousavy.camera.example I Buffer 1920 x 864 @ 1920
2023-11-08 20:24:17.361 31765-31861 VideoPipeline com.mrousavy.camera.example I ImageReader::onImageAvailable!
2023-11-08 20:24:17.427 31765-31861 VideoPipeline com.mrousavy.camera.example I ImageReader::onImageAvailable!
2023-11-08 20:24:17.464 31765-31861 VideoPipeline com.mrousavy.camera.example I ImageReader::onImageAvailable!
2023-11-08 20:24:17.496 31765-31861 VideoPipeline com.mrousavy.camera.example I ImageReader::onImageAvailable!
2023-11-08 20:24:17.535 31765-31861 VideoPipeline com.mrousavy.camera.example I ImageReader::onImageAvailable!
2023-11-08 20:24:17.563 31765-31861 VideoPipeline com.mrousavy.camera.example I ImageReader::onImageAvailable!
2023-11-08 20:24:17.598 31765-31861 VideoPipeline com.mrousavy.camera.example I ImageReader::onImageAvailable!
2023-11-08 20:24:17.640 31765-31861 VideoPipeline com.mrousavy.camera.example I ImageReader::onImageAvailable!
2023-11-08 20:24:17.664 31765-31861 VideoPipeline com.mrousavy.camera.example I ImageReader::onImageAvailable!
2023-11-08 20:24:17.712 31765-31861 VideoPipeline com.mrousavy.camera.example I ImageReader::onImageAvailable!
2023-11-08 20:24:17.733 31765-31861 VideoPipeline com.mrousavy.camera.example I ImageReader::onImageAvailable!
2023-11-08 20:24:17.774 31765-31861 VideoPipeline com.mrousavy.camera.example I ImageReader::onImageAvailable!
2023-11-08 20:24:17.800 31765-31861 VideoPipeline com.mrousavy.camera.example I ImageReader::onImageAvailable!
2023-11-08 20:24:17.839 31765-31861 VideoPipeline com.mrousavy.camera.example I ImageReader::onImageAvailable!
2023-11-08 20:24:17.868 31765-31861 VideoPipeline com.mrousavy.camera.example I ImageReader::onImageAvailable!
2023-11-08 20:24:17.913 31765-31861 VideoPipeline com.mrousavy.camera.example I ImageReader::onImageAvailable!
2023-11-08 20:24:17.938 31765-31861 VideoPipeline com.mrousavy.camera.example I ImageReader::onImageAvailable!
2023-11-08 20:24:17.978 31765-31861 VideoPipeline com.mrousavy.camera.example I ImageReader::onImageAvailable!
2023-11-08 20:24:18.011 31765-31861 VideoPipeline com.mrousavy.camera.example I ImageReader::onImageAvailable!
2023-11-08 20:24:18.035 31765-31861 VideoPipeline com.mrousavy.camera.example I ImageReader::onImageAvailable!
2023-11-08 20:24:18.067 31765-31861 VideoPipeline com.mrousavy.camera.example I ImageReader::onImageAvailable!
2023-11-08 20:24:18.106 31765-31861 VideoPipeline com.mrousavy.camera.example I ImageReader::onImageAvailable!
2023-11-08 20:24:18.136 31765-31861 VideoPipeline com.mrousavy.camera.example I ImageReader::onImageAvailable!
2023-11-08 20:24:18.170 31765-31861 VideoPipeline com.mrousavy.camera.example I ImageReader::onImageAvailable!
2023-11-08 20:24:18.209 31765-31861 VideoPipeline com.mrousavy.camera.example I ImageReader::onImageAvailable!
2023-11-08 20:24:18.250 31765-31861 VideoPipeline com.mrousavy.camera.example I ImageReader::onImageAvailable!
2023-11-08 20:24:18.278 31765-31861 VideoPipeline com.mrousavy.camera.example I ImageReader::onImageAvailable!
2023-11-08 20:24:18.303 31765-31861 VideoPipeline com.mrousavy.camera.example I ImageReader::onImageAvailable!
2023-11-08 20:24:18.341 31765-31861 VideoPipeline com.mrousavy.camera.example I ImageReader::onImageAvailable!
2023-11-08 20:24:18.342 31765-31861 Frame com.mrousavy.camera.example I Buffer 1920 x 864 @ 1920
2023-11-08 20:24:18.378 31765-31861 VideoPipeline com.mrousavy.camera.example I ImageReader::onImageAvailable!
2023-11-08 20:24:18.409 31765-31861 VideoPipeline com.mrousavy.camera.example I ImageReader::onImageAvailable!
2023-11-08 20:24:18.432 31765-31861 VideoPipeline com.mrousavy.camera.example I ImageReader::onImageAvailable!
2023-11-08 20:24:18.444 31765-31765 ViewRootIm...nActivity] com.mrousavy.camera.example I ViewPostIme pointer 0
2023-11-08 20:24:18.458 31765-31830 ReactNativeJS com.mrousavy.camera.example D state: BEGAN
2023-11-08 20:24:18.468 31765-31861 VideoPipeline com.mrousavy.camera.example I ImageReader::onImageAvailable!
2023-11-08 20:24:18.498 31765-31861 VideoPipeline com.mrousavy.camera.example I ImageReader::onImageAvailable!
2023-11-08 20:24:18.526 31765-31765 ViewRootIm...nActivity] com.mrousavy.camera.example I ViewPostIme pointer 1
2023-11-08 20:24:18.531 31765-31765 unknown:ReactNative com.mrousavy.camera.example W Can't cancel already finished gesture. Is a child View trying to start a gesture from an UP/CANCEL event?
2023-11-08 20:24:18.532 31765-31861 VideoPipeline com.mrousavy.camera.example I ImageReader::onImageAvailable!
2023-11-08 20:24:18.548 31765-31830 ReactNativeJS com.mrousavy.camera.example D state: ACTIVE
2023-11-08 20:24:18.550 31765-31830 ReactNativeJS com.mrousavy.camera.example D state: END
2023-11-08 20:24:18.550 31765-31830 ReactNativeJS com.mrousavy.camera.example I Taking photo...
2023-11-08 20:24:18.556 31765-31765 CameraView com.mrousavy.camera.example D Finding view 15...
2023-11-08 20:24:18.556 31765-31765 CameraView com.mrousavy.camera.example D Found view 15!
2023-11-08 20:24:18.557 31765-32033 CameraView.takePhoto com.mrousavy.camera.example I Taking photo... Options: {qualityPrioritization=speed, photoCodec=jpeg, quality=90.0, flash=off, enableShutterSound=false}
2023-11-08 20:24:18.557 31765-32033 CameraSession com.mrousavy.camera.example I Photo capture 0/3 - preparing capture request (2400x1080)...
2023-11-08 20:24:18.562 31765-32033 CameraSession com.mrousavy.camera.example I Photo capture 1/3 - starting capture...
2023-11-08 20:24:18.563 31765-31861 VideoPipeline com.mrousavy.camera.example I ImageReader::onImageAvailable!
2023-11-08 20:24:18.589 31765-31861 VideoPipeline com.mrousavy.camera.example I ImageReader::onImageAvailable!
2023-11-08 20:24:18.621 31765-31861 VideoPipeline com.mrousavy.camera.example I ImageReader::onImageAvailable!
2023-11-08 20:24:18.655 31765-31861 VideoPipeline com.mrousavy.camera.example I ImageReader::onImageAvailable!
2023-11-08 20:24:18.697 31765-31861 VideoPipeline com.mrousavy.camera.example I ImageReader::onImageAvailable!
2023-11-08 20:24:18.727 31765-31861 VideoPipeline com.mrousavy.camera.example I ImageReader::onImageAvailable!
2023-11-08 20:24:18.754 31765-31860 CameraSession com.mrousavy.camera.example I Photo Captured!
2023-11-08 20:24:18.755 31765-31860 CameraSession com.mrousavy.camera.example I Photo captured! 2400 x 1080
2023-11-08 20:24:18.756 31765-32033 CameraSession com.mrousavy.camera.example I Photo capture 2/3 complete - received metadata with timestamp 205459031779501
2023-11-08 20:24:18.756 31765-32033 CameraSession com.mrousavy.camera.example I Photo capture 3/3 complete - received 2400 x 1080 image.
2023-11-08 20:24:18.757 31765-32033 CameraView.takePhoto com.mrousavy.camera.example I Successfully captured 2400 x 1080 photo!
2023-11-08 20:24:18.795 31765-31861 VideoPipeline com.mrousavy.camera.example I ImageReader::onImageAvailable!
2023-11-08 20:24:18.807 31765-31774 .camera.example com.mrousavy.camera.example W Cleared Reference was only reachable from finalizer (only reported once)
2023-11-08 20:24:18.811 31765-31776 System com.mrousavy.camera.example W A resource failed to call Surface.release.
2023-11-08 20:24:18.812 31765-31776 System com.mrousavy.camera.example W A resource failed to call HardwareBuffer.close.
2023-11-08 20:24:18.812 31765-31776 libc com.mrousavy.camera.example A Fatal signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x0 in tid 31776 (FinalizerDaemon), pid 31765 (.camera.example)
2023-11-08 20:24:18.836 31765-32033 CameraView.takePhoto com.mrousavy.camera.example I Successfully saved photo to file! /data/user/0/com.mrousavy.camera.example/cache/mrousavy3036235101182437960.jpg
2023-11-08 20:24:18.837 31765-31830 ReactNativeJS com.mrousavy.camera.example I Media captured! {"isMirrored":true,"path":"/data/user/0/com.mrousavy.camera.example/cache/mrousavy3036235101182437960.jpg","isRawPhoto":false,"height":1080,"orientation":"landscape-left","width":2400}
2023-11-08 20:24:18.858 31765-31861 VideoPipeline com.mrousavy.camera.example I ImageReader::onImageAvailable!
2023-11-08 20:24:18.921 31765-31861 VideoPipeline com.mrousavy.camera.example I ImageReader::onImageAvailable!
2023-11-08 20:24:18.927 31765-31861 VideoPipeline com.mrousavy.camera.example I ImageReader::onImageAvailable!
2023-11-08 20:24:18.935 31765-31765 CameraView com.mrousavy.camera.example I Updating CameraSession...
2023-11-08 20:24:18.936 31765-31860 CameraSession com.mrousavy.camera.example I Updating CameraSession Configuration...
2023-11-08 20:24:18.936 31765-31860 CameraSession com.mrousavy.camera.example I Successfully updated CameraSession Configuration! isActive: false
2023-11-08 20:24:18.959 31765-31861 VideoPipeline com.mrousavy.camera.example I ImageReader::onImageAvailable!
2023-11-08 20:24:18.995 31765-31861 VideoPipeline com.mrousavy.camera.example I ImageReader::onImageAvailable!
2023-11-08 20:24:19.030 31765-31861 VideoPipeline com.mrousavy.camera.example I ImageReader::onImageAvailable!
2023-11-08 20:24:19.063 31765-31830 ReactNativeJS com.mrousavy.camera.example I Image loaded. Size: 1080x2256
2023-11-08 20:24:19.064 31765-31861 VideoPipeline com.mrousavy.camera.example I ImageReader::onImageAvailable!
2023-11-08 20:24:19.064 31765-31830 ReactNativeJS com.mrousavy.camera.example I media has loaded.
2023-11-08 20:24:19.094 31765-31861 VideoPipeline com.mrousavy.camera.example I ImageReader::onImageAvailable!
2023-11-08 20:24:19.099 31765-31780 CameraManagerGlobal com.mrousavy.camera.example I Camera 1 facing CAMERA_FACING_FRONT state now CAMERA_STATE_IDLE for client com.mrousavy.camera.example API Level 2User Id 0
---------------------------- PROCESS STARTED (32251) for package com.mrousavy.camera.example ----------------------------
2023-11-08 20:24:19.363 32249-32249 DEBUG pid-32249 A Cmdline: com.mrousavy.camera.example
2023-11-08 20:24:19.363 32249-32249 DEBUG pid-32249 A pid: 31765, tid: 31776, name: FinalizerDaemon >>> com.mrousavy.camera.example <<<
---------------------------- PROCESS ENDED (31765) for package com.mrousavy.camera.example ----------------------------
2023-11-08 20:24:19.520 1391-1458 WindowManager system_server E win=Window{6e5f636 u0 com.mrousavy.camera.example/com.mrousavy.camera.example.MainActivity EXITING} destroySurfaces: appStopped=false cleanupOnResume=false win.mWindowRemovalAllowed=true win.mRemoveOnExit=true win.mViewVisibility=0 caller=com.android.server.wm.ActivityRecord.destroySurfaces:6539 com.android.server.wm.ActivityRecord.destroySurfaces:6520 com.android.server.wm.WindowState.onExitAnimationDone:5987 com.android.server.wm.ActivityRecord$$ExternalSyntheticLambda10.accept:2 java.util.ArrayList.forEach:1262 com.android.server.wm.ActivityRecord.onAnimationFinished:8607 com.android.server.wm.ActivityRecord.postApplyAnimation:6252
---------------------------- PROCESS ENDED (32251) for package com.mrousavy.camera.example ----------------------------
Camera Device
{
"sensorOrientation": "landscape-left",
"hardwareLevel": "limited",
"maxZoom": 4,
"minZoom": 1,
"supportsLowLightBoost": true,
"neutralZoom": 1,
"physicalDevices": [
"wide-angle-camera"
],
"supportsFocus": true,
"supportsRawCapture": false,
"isMultiCam": false,
"name": "FRONT (1)",
"hasFlash": false,
"hasTorch": false,
"position": "front",
"id": "1"
}
Device
Galaxy S21
VisionCamera Version
3.6.4
Can you reproduce this issue in the VisionCamera Example app?
Yes, I can reproduce the same issue in the Example app here
Additional information
- [ ] I am using Expo
- [X] I have enabled Frame Processors (react-native-worklets-core)
- [X] I have read the Troubleshooting Guide
- [X] I agree to follow this project's Code of Conduct
- [X] I searched for similar issues in this repository and found none.
Same crash with vision camera v2 + reanimated v3.
Vision camera 2.15 + reanimated v2 works okay.
I got the same issue when using the vision camera v2 + reanimated v3. The reason I can't use vision camera v3 is that it support Android from 26 and above, which is only covert 93.7%
I love the vision camera v3 a lot but can't use it at the moment. If anyone know how to make the vision camera v2 work with reanimated v3, please let me know. Many thanks!
The latest version of vision camera v2 can work with reanimated v3: https://github.com/mrousavy/react-native-vision-camera/pull/1759
But takePhoto + frame processor seems to cause a crash.
Hello guys, I am able to build the camera v2 + reanimated v3. Below is the details of my package.json
"react": "18.2.0",
"react-native": "0.72.7",
"react-native-reanimated": "3.5.4",
"react-native-vision-camera": "2.16.5",
"vision-camera-code-scanner": "0.2.0"
Below is my script to build aab and apk:
"android-aab": "cd android && ./gradlew clean && rm -rf .gradle && ./gradlew bundleRelease && cd ../",
"android-apk": "cd android && ./gradlew clean && rm -rf .gradle && ./gradlew assembleRelease && cd ../",
Hope these can help people who have the same issue as me
Hey does this still crash? I made some changes here and there. Also error reporting should be better now.
Maybe I just fixed this in https://github.com/mrousavy/react-native-vision-camera/pull/2408 - could you test and verify?