react-native-vision-camera
react-native-vision-camera copied to clipboard
mrousavy
🐛 useSkiaFrameProcessor makes camera being blank
What's happening?
Hi again, apologies for bringing up multiple issues consecutively, but I'm really in need of your assistance.
Everything seems to be functioning properly except when I apply useSkiaFrameProcessor to the camera. The camera screen goes blank.
I tried the exact same process in the example app, and it worked as expected. I've compared the setup in my app with the example app, and everything appears to be identical. It's possible that I've made a mistake somewhere, but I'm reaching out to see if you have any insights into what I might have done wrong.
Reproduceable Code
pacakge.json
"react-native-vision-camera": "^4.0.3",
"react-native-worklets-core": "^1.2.0",
"@shopify/react-native-skia": "^1.2.3",
const frameProcessor = useSkiaFrameProcessor(frame => {
'worklet';
frame.render();
const centerX = frame.width / 2;
const centerY = frame.height / 2;
const rect = Skia.XYWHRect(centerX, centerY, 150, 150);
const paint = Skia.Paint();
paint.setColor(Skia.Color('red'));
frame.drawRect(rect, paint);
}, []);
const format = useCameraFormat(device, [
{fps: DEFAULT_FPS},
{photoAspectRatio: camera_ratio},
{photoResolution: 'max'},
{photoHdr: true},
{videoAspectRatio: camera_ratio},
{videoResolution: 'max'},
{videoHdr: true},
]);
<ReanimatedCameraView
ref={camera}
device={device}
isActive={camera_isActive}
style={styles.container}
photo={true}
video={true}
audio={false}
orientation={cameraOrientation}
animatedProps={animatedProps}
format={format}
resizeMode={'cover'}
frameProcessor={frameProcessor}
photoHdr={format?.supportsPhotoHdr}
videoHdr={format?.supportsVideoHdr}
photoQualityBalance={'quality'}
fps={fps}
onError={onError}
/>
Relevant log output
17:16:20.390: [info] 📸 VisionCamera.didSetProps(_:): Updating 30 props: [onInitialized, alignItems, cameraId, enableBufferCompression, preview, onStarted, onCodeScanned, collapsable, justifyContent, photoHdr, position, isActive, video, resizeMode, fps, onStopped, onError, onViewReady, enableFrameProcessor, videoHdr, format, orientation, audio, height, width, zoom, photoQualityBalance, exposure, photo, onShutter]
17:16:20.392: [info] 📸 VisionCamera.configure(_:): configure { ... }: Waiting for lock...
17:16:20.401: [info] 📸 VisionCamera.configure(_:): configure { ... }: Updating CameraSession Configuration... Difference(inputChanged: true, outputsChanged: true, videoStabilizationChanged: true, orientationChanged: true, formatChanged: true, sidePropsChanged: true, torchChanged: true, zoomChanged: true, exposureChanged: true, audioSessionChanged: true, locationChanged: true)
17:16:20.401: [info] 📸 VisionCamera.configureDevice(configuration:): Configuring Input Device...
17:16:20.401: [info] 📸 VisionCamera.configureDevice(configuration:): Configuring Camera com.apple.avfoundation.avcapturedevice.built-in_video:6...
17:16:20.409: [info] 📸 VisionCamera.configureDevice(configuration:): Successfully configured Input Device!
17:16:20.409: [info] 📸 VisionCamera.configureOutputs(configuration:): Configuring Outputs...
17:16:20.409: [info] 📸 VisionCamera.configureOutputs(configuration:): Adding Photo output...
17:16:20.416: [info] 📸 VisionCamera.configureOutputs(configuration:): Adding Video Data output...
17:16:20.418: [info] 📸 VisionCamera.configureOutputs(configuration:): Successfully configured all outputs!
17:16:20.419: [info] 📸 VisionCamera.configureFormat(configuration:device:): Configuring Format (2112x1188 | [email protected] (ISO: 32.0..3072.0))...
17:16:20.422: [info] 📸 VisionCamera.configureFormat(configuration:device:): Successfully configured Format!
17:16:20.427: [info] 📸 VisionCamera.getPixelFormat(for:): Available Pixel Formats: ["420v", "420f", "BGRA"], finding best match... (pixelFormat="yuv", enableHdr={false}, enableBufferCompression={false})
17:16:20.427: [info] 📸 VisionCamera.getPixelFormat(for:): Using PixelFormat: 420f...
'user login status: ', true
17:16:20.446: [info] 📸 VisionCamera.onCameraStarted(): Camera started!
17:16:20.446: [info] 📸 VisionCamera.onSessionInitialized(): Camera initialized!
17:16:20.446: [info] 📸 VisionCamera.configure(_:): Beginning AudioSession configuration...
17:16:20.446: [info] 📸 VisionCamera.configureAudioSession(configuration:): Configuring Audio Session...
17:16:20.446: [info] 📸 VisionCamera.configure(_:): Beginning Location Output configuration...
17:16:20.450: [info] 📸 VisionCamera.configure(_:): Committed AudioSession configuration!
17:16:20.451: [info] 📸 VisionCamera.didSetProps(_:): Updating 1 props: [isActive]
17:16:20.451: [info] 📸 VisionCamera.configure(_:): configure { ... }: Waiting for lock...
17:16:20.453: [info] 📸 VisionCamera.configure(_:): configure { ... }: Updating CameraSession Configuration... Difference(inputChanged: false, outputsChanged: false, videoStabilizationChanged: false, orientationChanged: false, formatChanged: false, sidePropsChanged: false, torchChanged: false, zoomChanged: false, exposureChanged: false, audioSessionChanged: false, locationChanged: false)
17:16:20.615: [info] 📸 VisionCamera.configure(_:): Finished Location Output configuration!
{ isRecordingWithFlash: false, averageFpsSamples: [] }
17:16:20.663: [info] 📸 VisionCamera.didSetProps(_:): Updating 1 props: [orientation]
17:16:20.663: [info] 📸 VisionCamera.configure(_:): configure { ... }: Waiting for lock...
17:16:20.664: [info] 📸 VisionCamera.configure(_:): configure { ... }: Updating CameraSession Configuration... Difference(inputChanged: false, outputsChanged: false, videoStabilizationChanged: false, orientationChanged: true, formatChanged: false, sidePropsChanged: false, torchChanged: false, zoomChanged: false, exposureChanged: false, audioSessionChanged: false, locationChanged: false)
17:19:43.196: [info] 📸 VisionCamera.didSetProps(_:): Updating 30 props: [onInitialized, alignItems, cameraId, enableBufferCompression, preview, onStarted, onCodeScanned, collapsable, justifyContent, photoHdr, position, isActive, video, resizeMode, fps, onStopped, onError, onViewReady, enableFrameProcessor, videoHdr, format, orientation, audio, height, width, zoom, photoQualityBalance, exposure, photo, onShutter]
17:19:43.196: [info] 📸 VisionCamera.configure(_:): configure { ... }: Waiting for lock...
17:19:43.197: [info] 📸 VisionCamera.configure(_:): configure { ... }: Updating CameraSession Configuration... Difference(inputChanged: true, outputsChanged: true, videoStabilizationChanged: true, orientationChanged: true, formatChanged: true, sidePropsChanged: true, torchChanged: true, zoomChanged: true, exposureChanged: true, audioSessionChanged: true, locationChanged: true)
17:19:43.197: [info] 📸 VisionCamera.configureDevice(configuration:): Configuring Input Device...
17:19:43.197: [info] 📸 VisionCamera.configureDevice(configuration:): Configuring Camera com.apple.avfoundation.avcapturedevice.built-in_video:6...
17:19:43.206: [info] 📸 VisionCamera.configureDevice(configuration:): Successfully configured Input Device!
17:19:43.206: [info] 📸 VisionCamera.configureOutputs(configuration:): Configuring Outputs...
17:19:43.206: [info] 📸 VisionCamera.configureOutputs(configuration:): Adding Photo output...
17:19:43.209: [info] 📸 VisionCamera.configureOutputs(configuration:): Adding Video Data output...
17:19:43.210: [info] 📸 VisionCamera.configureOutputs(configuration:): Successfully configured all outputs!
17:19:43.210: [info] 📸 VisionCamera.configureFormat(configuration:device:): Configuring Format (2112x1188 | [email protected] (ISO: 32.0..3072.0))...
17:19:43.212: [info] 📸 VisionCamera.configureFormat(configuration:device:): Successfully configured Format!
17:19:43.212: [info] 📸 VisionCamera.getPixelFormat(for:): Available Pixel Formats: ["420v", "420f", "BGRA"], finding best match... (pixelFormat="yuv", enableHdr={false}, enableBufferCompression={false})
17:19:43.212: [info] 📸 VisionCamera.getPixelFormat(for:): Using PixelFormat: 420f...
17:19:43.215: [info] 📸 VisionCamera.onSessionInitialized(): Camera initialized!
17:19:43.215: [info] 📸 VisionCamera.configure(_:): Beginning AudioSession configuration...
17:19:43.215: [info] 📸 VisionCamera.configureAudioSession(configuration:): Configuring Audio Session...
17:19:43.215: [info] 📸 VisionCamera.configure(_:): Beginning Location Output configuration...
17:19:43.216: [info] 📸 VisionCamera.configure(_:): Committed AudioSession configuration!
17:19:43.218: [info] 📸 VisionCamera.configure(_:): Finished Location Output configuration!
17:21:44.867: [info] 📸 VisionCamera.didSetProps(_:): Updating 1 props: [orientation]
17:21:44.867: [info] 📸 VisionCamera.configure(_:): configure { ... }: Waiting for lock...
17:21:44.868: [info] 📸 VisionCamera.configure(_:): configure { ... }: Updating CameraSession Configuration... Difference(inputChanged: false, outputsChanged: false, videoStabilizationChanged: false, orientationChanged: true, formatChanged: false, sidePropsChanged: false, torchChanged: false, zoomChanged: false, exposureChanged: false, audioSessionChanged: false, locationChanged: false)
17:21:44.904: [info] 📸 VisionCamera.didSetProps(_:): Updating 1 props: [orientation]
17:21:44.904: [info] 📸 VisionCamera.configure(_:): configure { ... }: Waiting for lock...
17:21:44.904: [info] 📸 VisionCamera.configure(_:): configure { ... }: Updating CameraSession Configuration... Difference(inputChanged: false, outputsChanged: false, videoStabilizationChanged: false, orientationChanged: true, formatChanged: false, sidePropsChanged: false, torchChanged: false, zoomChanged: false, exposureChanged: false, audioSessionChanged: false, locationChanged: false)
17:21:44.997: [info] 📸 VisionCamera.didSetProps(_:): Updating 1 props: [orientation]
17:21:44.997: [info] 📸 VisionCamera.configure(_:): configure { ... }: Waiting for lock...
17:21:44.997: [info] 📸 VisionCamera.configure(_:): configure { ... }: Updating CameraSession Configuration... Difference(inputChanged: false, outputsChanged: false, videoStabilizationChanged: false, orientationChanged: true, formatChanged: false, sidePropsChanged: false, torchChanged: false, zoomChanged: false, exposureChanged: false, audioSessionChanged: false, locationChanged: false)
17:21:45.382: [info] 📸 VisionCamera.didSetProps(_:): Updating 1 props: [orientation]
17:21:45.382: [info] 📸 VisionCamera.configure(_:): configure { ... }: Waiting for lock...
17:21:45.383: [info] 📸 VisionCamera.configure(_:): configure { ... }: Updating CameraSession Configuration... Difference(inputChanged: false, outputsChanged: false, videoStabilizationChanged: false, orientationChanged: true, formatChanged: false, sidePropsChanged: false, torchChanged: false, zoomChanged: false, exposureChanged: false, audioSessionChanged: false, locationChanged: false)
(ADVICE) View #247 of type RCTView has a shadow set but cannot calculate shadow efficiently. Consider setting a solid background color to fix this, or apply the shadow to a more specific component.
(ADVICE) View #247 of type RCTView has a shadow set but cannot calculate shadow efficiently. Consider setting a solid background color to fix this, or apply the shadow to a more specific component.
[I] <MMKV_OSX.cpp:99::setIsInBackground> g_isInBackground:0
[I] <libMMKV.mm:313::+[MMKV didBecomeActive]> isInBackground:0
17:21:49.806: [info] 📸 VisionCamera.didSetProps(_:): Updating 1 props: [isActive]
17:21:49.806: [info] 📸 VisionCamera.configure(_:): configure { ... }: Waiting for lock...
17:21:49.808: [info] 📸 VisionCamera.configure(_:): configure { ... }: Updating CameraSession Configuration... Difference(inputChanged: false, outputsChanged: false, videoStabilizationChanged: false, orientationChanged: false, formatChanged: false, sidePropsChanged: false, torchChanged: false, zoomChanged: false, exposureChanged: false, audioSessionChanged: false, locationChanged: false)
17:21:50.253: [info] 📸 VisionCamera.onCameraStarted(): Camera started!
17:21:50.289: [info] 📸 VisionCamera.didSetProps(_:): Updating 1 props: [orientation]
17:21:50.289: [info] 📸 VisionCamera.configure(_:): configure { ... }: Waiting for lock...
17:21:50.290: [info] 📸 VisionCamera.configure(_:): configure { ... }: Updating CameraSession Configuration... Difference(inputChanged: false, outputsChanged: false, videoStabilizationChanged: false, orientationChanged: true, formatChanged: false, sidePropsChanged: false, torchChanged: false, zoomChanged: false, exposureChanged: false, audioSessionChanged: false, locationChanged: false)
<<<< FigCaptureSessionRemote >>>> Fig assert: "err == 0 " at bail (FigCaptureSessionRemote.m:866) - (err=-12784)
[I] <MMKV_OSX.cpp:99::setIsInBackground> g_isInBackground:1
[I] <libMMKV.mm:308::+[MMKV didEnterBackground]> isInBackground:1
17:21:51.390: [info] 📸 VisionCamera.didSetProps(_:): Updating 1 props: [isActive]
17:21:51.390: [info] 📸 VisionCamera.configure(_:): configure { ... }: Waiting for lock...
17:21:51.390: [info] 📸 VisionCamera.configure(_:): configure { ... }: Updating CameraSession Configuration... Difference(inputChanged: false, outputsChanged: false, videoStabilizationChanged: false, orientationChanged: false, formatChanged: false, sidePropsChanged: false, torchChanged: false, zoomChanged: false, exposureChanged: false, audioSessionChanged: false, locationChanged: false)
[I] <libMMKV.mm:301::-[MMKV onMemoryWarning]> cleaning on memory warning mmkv.default
[I] <MMKV.cpp:307::clearMemoryCache> clearMemoryCache [mmkv.default]
[I] <MemoryFile.cpp:104::close> closing fd[0x15], /var/mobile/Containers/Data/Application/A90E070A-F05A-469B-9785-B47EDD2E7D22/Documents/mmkv/mmkv.default
WARNING: Logging before InitGoogleLogging() is written to STDERR
I0507 17:21:51.708926 1843605504 JSIExecutor.cpp:348] Memory warning (pressure level: TRIM_MEMORY_RUNNING_CRITICAL) received by JS VM, running a GC
17:21:53.775: [info] 📸 VisionCamera.didSetProps(_:): Updating 1 props: [orientation]
17:21:53.775: [info] 📸 VisionCamera.configure(_:): configure { ... }: Waiting for lock...
17:21:53.776: [info] 📸 VisionCamera.configure(_:): configure { ... }: Updating CameraSession Configuration... Difference(inputChanged: false, outputsChanged: false, videoStabilizationChanged: false, orientationChanged: true, formatChanged: false, sidePropsChanged: false, torchChanged: false, zoomChanged: false, exposureChanged: false, audioSessionChanged: false, locationChanged: false)
Camera Device
{
"formats": [],
"isMultiCam": true,
"maxZoom": 123.75,
"hasTorch": true,
"id": "com.apple.avfoundation.avcapturedevice.built-in_video:6",
"minFocusDistance": 12,
"name": "Back Dual Wide Camera",
"minExposure": -8,
"hasFlash": true,
"minZoom": 1,
"supportsRawCapture": false,
"neutralZoom": 2,
"physicalDevices": [
"ultra-wide-angle-camera",
"wide-angle-camera"
],
"supportsFocus": true,
"sensorOrientation": "landscape-right",
"maxExposure": 8,
"supportsLowLightBoost": false,
"hardwareLevel": "full",
"position": "back"
}
Device
iPhont 11 IOS17.4.1
VisionCamera Version
4.0.3
Can you reproduce this issue in the VisionCamera Example app?
No, I cannot reproduce the issue in the Example app
Additional information
- [ ] I am using Expo
- [X] I have enabled Frame Processors (react-native-worklets-core)
- [X] I have read the Troubleshooting Guide
- [X] I agree to follow this project's Code of Conduct
- [X] I searched for similar issues in this repository and found none.
