🐛 Camera freezes on first frame when running frameProcessors on Galaxy S8, Android 9

[joncys]

What's happening?

So I am testing frame processors on SAMSUNG Galaxy S8, running Android 9.0. The previous version works fine (v2), but on v3 the Camera freezes on the first frame, when (and only when) running frame processors. If I comment out the frameProcessor prop, it works fine. So in theory I am able to have the worklet initialized, it's only when I actually try to use it and bind it to the camera, it freezes and stops the camera from working. Also the console.logs from the frame processor are not being logged, so the worklet does not seem to be running as well (or binded).

I tried running the example app and the same thing happens over there as well. Will be pasting both the example app and my app logs. Both the logs are just launching the app and it arriving on the camera preview screen. I have dealt with permissions beforehand. I open the apps up and then the camera freezes, so I wait a bit, pressing on the screen so that it doesn't lock the phone, and then I open up the app list to close the app and end the session.

I'm super not familiar with native development, so I might be using the wrong lingo or something and this is quite as much context as I can possibly provide.

Reproduceable Code

// this is for my app, but nothing really special here, so I would
// assume that the problem is somewhere deep down

const frameProcessor = useFrameProcessor((frame) => {
    'worklet'
    console.log('hello')
    return null
  }, [])

  const device = useCameraDevice('back')

  if (device == null) return <Text>No camera</Text>
  return <Camera
    style={{
      flex: 1,
      width: '100%',
    }}
    device={device}
    isActive={true}
    frameProcessor={frameProcessor}
  />

Relevant log output

It says, that my logs are too long, I've created a pastebin
https://pastebin.com/aEezEEZi

Sorry, I'm not great at native, I don't really understand what logs are relevant
There are no real faults or exceptions (nothing too read), but a few Java errors
are being logged with the INFO status

Nothing interesting on the React Native logs

Camera Device

{
  "sensorOrientation": "landscape-right",
  "hardwareLevel": "full",
  "maxZoom": 8,
  "minZoom": 1,
  "maxExposure": 8,
  "supportsLowLightBoost": false,
  "neutralZoom": 1,
  "physicalDevices": [
    "wide-angle-camera"
  ],
  "supportsFocus": true,
  "supportsRawCapture": true,
  "isMultiCam": false,
  "minExposure": -8,
  "name": "BACK (0)",
  "hasFlash": true,
  "hasTorch": true,
  "position": "back",
  "id": "0"
}

Device

SAMSUNG Galaxy S8

VisionCamera Version

3.6.16

Can you reproduce this issue in the VisionCamera Example app?

Yes, I can reproduce the same issue in the Example app here

Additional information

• [X] I am using Expo
• [X] I have enabled Frame Processors (react-native-worklets-core)
• [X] I have read the Troubleshooting Guide
• [X] I agree to follow this project's Code of Conduct
• [X] I searched for similar issues in this repository and found none.

[Wondae-code]

Is there any progress??

I'm using similar device (Samsung Galaxy Note 8) and Android 9. and I have a same problem.

If you solved this problem, please share any Information.

[stewartiee]

Experiencing the same on a new project, on a Pixel 3. First frame shows for a second, and then the app crashes.

If you remove the frameProcessor on the Camera component, then it shows fine with no crash (and no frame processors).

[yuraxdrumz]

experiencing the same issue on a one plus nord with android 11

[gdaitis]

same exact problem on the same exact device. Has anyone taken a look at this?

[fazzysyed]

  const format = React.useMemo(() => {
    const desiredWidth = 1920;
    const desiredHeight = 1080;
    if (device) {
      for (let index = 0; index < device.formats.length; index++) {
        const format = device.formats[index];
        if (format) {
          if (
            format.videoWidth == desiredWidth &&
            format.videoHeight == desiredHeight
          ) {
            return format;
          }
        }
      }
    }
    return undefined;
  }, [device?.formats]);

I had the same problem, but when I assigned the correct format, the camera worked fine

[joncys]

@fazzysyed Setting the format helps a little in my case because it goes straight to an

 Camera.onError(unknown/unknown): [unknown/unknown] CAMERA_ERROR (3): waitUntilIdle:1339: Camera 0: Error waiting to drain: Connection timed out (-110) [unknown/unknown: [unknown/unknown] CAMERA_ERROR (3): waitUntilIdle:1339: Camera 0: Error waiting to drain: Connection timed out (-110)]

camera error instead of just hanging on a black screen.

But the camera / plugin still does not seem to be working as expected.

[masanchez-hyve]

I'm experiencing the same issue with the POCO F3 Pro.

[mrousavy]

Thanks for sharing the logs @joncys - unfortunately I cannot access pastebin rn so I'll take another looks later, as an alternative you could use gist.github.com for hosting the logs :)

[touchmarine]

I had the same issue, but it works with pixelFormat="yuv". (Galaxy S8+, Android 9)

This is already the default pixel format on main (https://github.com/mrousavy/react-native-vision-camera/commit/11eb1f68e4912309589855ee81edea12f5261df4, https://github.com/mrousavy/react-native-vision-camera/commit/d64fc08e167a6f2787f6deccfb3ba0caf16557b0) and used by the codeScanner.

[gdaitis]

@mrousavy here is the content from @joncys pastebin: https://gist.github.com/gdaitis/47f62b5b994b28f5edc55e7481d1f71e

[mrousavy]

Hmm, there's nothing that's particularily interesting in those logs.... 🤔

[joncys]

Tried running it with 3.9.0 beta and pixelFormat={'yuv'} set. Now it does not freeze, but the whole app crashes, maybe that is something that you can work with.

Sharing logs for the crash here: https://gist.github.com/joncys/303fcbadf05675fbf70ac001a7f29183

[mrousavy]

2024-02-01 17:29:19.942 10876-10974 HermesVM                lt.joncys.example                    I  Compiling JS failed: 1:1:invalid empty parentheses '( )',  Buffer size 3 starts with: 280a29 and has protection mode(s): rw-p
2024-02-01 17:29:19.981 10876-10974 VideoPipeline           lt.joncys.example                    E  FrameProcessor/ImageReader pipeline threw an error!
                                                                                                    com.facebook.jni.CppException: Compiling JS failed: 1:1:invalid empty parentheses '( )' Buffer size 3 starts with: 280a29 and has protection mode(s): rw-p

@joncys This is a different issue and has been reported before a lot - it is basically some compatibility issue in your Babel plugins (Worklets vs Reanimated). Make sure to clean cache, update your versions, and try again.

Once that is gone let me know if 3.9 works :)

[marcshilling]

All my users with POCO F2/F3 devices reporting this freeze. Android 12 and 13, maybe others. I'm already using pixelFormat="yuv" so can't be that. Unfortunately I don't have this device to test with/reproduce.

[mrousavy]

@marcshilling are you on 3.9-beta.0?

[marcshilling]

@mrousavy nah 3.8.2

[brunoobatista]

I had this issue with the frameProcessor on a Samsung A8 with Android 9, but when I tested the RNVC example application, where there is the implementation of frameProcessors, it works fine. I tried applying what's there in an Expo project as well, but the same problem of freezing the camera upon opening occurs. When I created the project without Expo for testing, it worked normally.

rnvc: 3.8.2 expo: 49-50 rn: 0.72-0.73

logs error

[krzychuuu132]

3.9.0-beta.2 the same issue ://

[rampantslug]

I have a Poco F3 and using a frameprocessor that freezes on first frame. I am using scanOCR from vision-camera-ocr. I'll do some more tests but if I just log inside the frameprocessor it is fine. Cant be sure if it is the scanOCR call or the attempt to run in js using - Worklets.createRunInJsFn() I'm on 3.8.2 I'll test using a worklet without processing OCR and report back

[rampantslug]

Yep. Any attempt (just attempting to console.log frame details) to use a frameprocessor with a worklet causes it to freeze after first frame.

[Thormeard]

Same issue. Empty frameProcessor (with just console.log), freezing. If I comment out frameProcessor prop, it works fine.

Built-in QR scanner is working (if this matter)

OnePlus Nord AC2003, Android 12, kernel 4.19.157-perf+

vision camera 3.9.1 RN 73.4 react-native-worklets-core 0.3.0

[DanInDev]

Just had a very similiar issue. I fixed it by setting:

photo={true}

Also saw in a thread similiar that:

video={true}

Also worked for some.

[Thormeard]

Just had a very similiar issue. I fixed it by setting:

photo={true}

Also saw in a thread similiar that:

video={true}

Also worked for some.

Can confirm, photo={true} fixes the freezing issue for me (video don't do anything)

[Kavosh-m]

Same issue with some SAMSUNG devices.

const format = useCameraFormat(device, [{ videoResolution: { width: 1920, height: 1080 } }, { fps: 30 }])

<Camera
                        key={'camera1'}
                        style={StyleSheet.absoluteFill}
                        isActive={isActive}
                        device={device}
                        format={format}
                        frameProcessor={frameProcessor}
                        pixelFormat="yuv"
                        onStarted={() => {
                            console.log('camera is ready');
                            updateCropRegion();
                        }}
                        onError={error => {
                            logToSentry(error, true, 'Camera runtime error happend');
                        }}
                    />

[mrousavy]

Hey! I just released a new V4 beta (v4.0.0-beta.7) where I fixed a bunch of issues! 💪🚀 Can you test that and see if that fixes the issue for you? 😅

yarn add react-native-vision-camera@beta

You might need to increase the compileSdk to the newest Android SDK (34) if you are behind.

[mrousavy]

If that bug got fixed in latest V4 beta, please consider 💖 sponsoring me on GitHub 💖 so I can keep maintaining this library, fixing bugs and building new features! :)