mrousavy
🐛 Black screen no some devices using Frame Processor
What's happening?
Using frame processor shows black image on emulator and some devices (can't reproduce the real devices, read it from android play store reviews).
With frameProcessor
Without frameProcessor
Note: The image is black when negative is false and white when negative is true, meaning the shader is applied correctly, but the image isn't displayed. Also relevant, the resulting image taken is correct and shows the real picture. It's just the live preview with frame processor that isn't working.
Reproduceable Code
using following shader:
const invertColorsFilter = Skia.RuntimeEffect.Make(`
uniform shader image;
half4 main(vec2 pos) {
vec4 color = image.eval(pos);
return vec4((1.0 - color).rgb, 1.0);
}
`)
const shaderBuilder = Skia.RuntimeShaderBuilder(invertColorsFilter)
const imageFilter = Skia.ImageFilter.MakeRuntimeShader(shaderBuilder, null, null)
const paint = Skia.Paint()
const defaultPaint = Skia.Paint()
paint.setImageFilter(imageFilter)
set here:
const frameProcessor = useSkiaFrameProcessor(frame => {
'worklet'
if (negativeColors) {
frame.render(paint)
} else {
frame.render(defaultPaint)
}
}, [paint, defaultPaint, negativeColors])
to my component:
<ReanimatedCamera
style={StyleSheet.absoluteFill}
device={device}
isActive={isActive}
ref={camera}
onInitialized={onInitialized}
onError={onError}
onStarted={() => console.log('Camera started!')}
onStopped={() => console.log('Camera stopped!')}
onPreviewStarted={() => console.log('Preview started!')}
onPreviewStopped={() => console.log('Preview stopped!')}
onOutputOrientationChanged={(o) => console.log(`Output orientation changed to ${o}!`)}
onPreviewOrientationChanged={(o) => console.log(`Preview orientation changed to ${o}!`)}
onUIRotationChanged={(degrees) => console.log(`UI Rotation changed: ${degrees}°`)}
format={format}
fps={30}
photoHdr={photoHdr}
photoQualityBalance="speed"
lowLightBoost={device.supportsLowLightBoost}
enableZoomGesture={false}
animatedProps={cameraAnimatedProps}
exposure={0}
enableFpsGraph={false}
outputOrientation="device"
photo={true}
video={false}
audio={false}
enableLocation={false}
frameProcessor={frameProcessor}
/>
### Relevant log output
```shell
2025-01-13 09:30:53.407 5114-5114 unknown:co...agerHelper com.klim.klimfilmscanner E Unhandled SoftException
com.facebook.react.bridge.ReactNoCrashSoftException: Cannot get UIManager because the context doesn't contain an active CatalystInstance.
at com.facebook.react.uimanager.UIManagerHelper.getUIManager(UIManagerHelper.java:82)
at com.facebook.react.uimanager.UIManagerHelper.getEventDispatcher(UIManagerHelper.java:134)
at com.facebook.react.uimanager.UIManagerHelper.getEventDispatcherForReactTag(UIManagerHelper.java:112)
at com.mrousavy.camera.react.CameraView_EventsKt.sendEvent(CameraView+Events.kt:173)
at com.mrousavy.camera.react.CameraView_EventsKt.invokeOnPreviewStarted(CameraView+Events.kt:46)
at com.mrousavy.camera.react.CameraView$createPreviewView$1$1.invoke(CameraView.kt:301)
at com.mrousavy.camera.react.CameraView$createPreviewView$1$1.invoke(CameraView.kt:294)
at com.mrousavy.camera.react.CameraView.createPreviewView$lambda$2$lambda$1(CameraView.kt:294)
at com.mrousavy.camera.react.CameraView.$r8$lambda$ZoRs9LXhNa6nHYEs80y02k4qaAQ(Unknown Source:0)
at com.mrousavy.camera.react.CameraView$$ExternalSyntheticLambda1.onChanged(D8$$SyntheticClass:0)
at androidx.lifecycle.LiveData.considerNotify(LiveData.java:133)
at androidx.lifecycle.LiveData.dispatchingValue(LiveData.java:151)
at androidx.lifecycle.LiveData.setValue(LiveData.java:309)
at androidx.lifecycle.MutableLiveData.setValue(MutableLiveData.java:50)
at androidx.lifecycle.LiveData$1.run(LiveData.java:93)
at android.os.Handler.handleCallback(Handler.java:959)
at android.os.Handler.dispatchMessage(Handler.java:100)
at android.os.Looper.loopOnce(Looper.java:232)
at android.os.Looper.loop(Looper.java:317)
at android.app.ActivityThread.main(ActivityThread.java:8705)
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:580)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:886)
2025-01-13 09:30:53.537 5114-5114 EGL_emulation com.klim.klimfilmscanner E eglQueryContext 32c0 EGL_BAD_ATTRIBUTE
2025-01-13 09:30:53.538 5114-5114 EGL_emulation com.klim.klimfilmscanner E tid 5114: eglQueryContext(2161): error 0x3004 (EGL_BAD_ATTRIBUTE)
2025-01-13 09:30:53.580 5114-5114 Surface com.klim.klimfilmscanner E freeAllBuffers: 1 buffers were freed while being dequeued!
2025-01-13 09:30:54.593 5114-5114 Surface com.klim.klimfilmscanner E getSlotFromBufferLocked: unknown buffer: 0x757fa93df0b0
2025-01-13 09:30:54.603 5114-5114 klimfilmscanner com.klim.klimfilmscanner E [SurfaceTexture-0-5114-22] updateAndRelease: EGLConsumer is not attached to an OpenGL ES context
2025-01-13 09:30:55.175 5114-5114 klimfilmscanner com.klim.klimfilmscanner E [SurfaceTexture-0-5114-22] updateAndRelease: EGLConsumer is not attached to an OpenGL ES context
2025-01-13 09:30:55.227 5114-5114 klimfilmscanner com.klim.klimfilmscanner E [SurfaceTexture-0-5114-22] updateAndRelease: EGLConsumer is not attached to an OpenGL ES context
2025-01-13 09:30:59.141 5114-5114 klimfilmscanner com.klim.klimfilmscanner E [SurfaceTexture-0-5114-22] updateAndRelease: EGLConsumer is not attached to an OpenGL ES context
2025-01-13 09:31:02.147 5114-5114 klimfilmscanner com.klim.klimfilmscanner E [SurfaceTexture-0-5114-22] updateAndRelease: EGLConsumer is not attached to an OpenGL ES context
2025-01-13 09:31:02.521 5114-5114 klimfilmscanner com.klim.klimfilmscanner E [SurfaceTexture-0-5114-22] updateAndRelease: EGLConsumer is not attached to an OpenGL ES context
Camera Device
{
"formats": [],
"sensorOrientation": "landscape-left",
"hardwareLevel": "limited",
"maxZoom": 1,
"minZoom": 1,
"maxExposure": 6,
"supportsLowLightBoost": false,
"neutralZoom": 1,
"physicalDevices": [
"ultra-wide-angle-camera"
],
"supportsFocus": true,
"supportsRawCapture": false,
"isMultiCam": false,
"minFocusDistance": 0,
"minExposure": -6,
"name": "10 (BACK) androidx.camera.camera2",
"hasFlash": false,
"hasTorch": false,
"position": "back",
"id": "10"
}
Device
android emulator with API 35 and "VirturalScene"
VisionCamera Version
4.6.3 with skia 1.7.0
Can you reproduce this issue in the VisionCamera Example app?
No, I cannot reproduce the issue in the Example app
Additional information
- [X] I am using Expo
- [X] I have enabled Frame Processors (react-native-worklets-core)
- [X] I have read the Troubleshooting Guide
- [X] I agree to follow this project's Code of Conduct
- [X] I searched for similar issues in this repository and found none.
Guten Tag, Hans here 🍻.
Thanks for the detailed report! It looks like you're experiencing an issue with the frame processor displaying a black image on some devices. However, mrousavy will need more relevant logs to investigate this further, particularly runtime logs from adb logcat.
You can gather those by running adb logcat in your terminal while the app is open on the device or emulator. This will help us troubleshoot the issue more effectively.
Feel free to update the issue once you have the logs!
Note: If you think I made a mistake, please ping
@mrousavyto take a look.
![maintenance-hans[bot] avatar](https://avatars.githubusercontent.com/in/918494?v=4 "Published by a pub.dev verified publisher"){kind=small}
@mrousavy updated the issue with relevant adb logs pointing to the exception
@mrousavy According wcandillon fromSkia it's an issue related to devices who are using Vulkan and Angle for OpenGL support, including a lot of samsung devices. It was fixed in skia 1.7.7, maybe updating skia from 1.4.2 to 1.7.7 (or higher) would fix all those black screen issues?
- Please stop pinging me, I am not free support.
- Skia is a
*peerDependency - it is not pinned to a version. If you install 1.7.7., VisionCamera will use Skia 1.7.7. - not 1.4.2.
In some tests I made black screen is related to canvas translation: https://github.com/mrousavy/react-native-vision-camera/blob/b194ef17902d8702f584b8f0a5dee0f5c96fceb1/package/src/skia/useSkiaFrameProcessor.ts#L98-L99
If I use canvas.translate(0, frame.width) black screen is solved but preview is inverted (left is right and right is left).
Also, drawing on frame is buggy too because seems the frame is -90deg rotated while on IOS it works fine
EDIT: To fix black screen + mirroring on android I changed this: https://github.com/mrousavy/react-native-vision-camera/blob/b194ef17902d8702f584b8f0a5dee0f5c96fceb1/package/src/skia/useSkiaFrameProcessor.ts#L98-L99
To this:
canvas.scale(1, -1)
canvas.translate(frame.height, -frame.width)
canvas.rotate(90, 0, 0)
But this makes drawing on frame with skia 90deg rotated...
Nothing works on my side, also I logged and am getting 'landscape-right' by default, not landscape-left. Black screen on some samsung devices like Pixel 9 with api 35 and crash on samsung s8.
I also experience crashes using the Pixel 7 Pro and 6 with API 35 and frame processors, but I haven't been able to resolve the issue. Has something worked for anyone?
I also experience crashes using the Pixel 7 Pro and 6 with API 35 and frame processors, but I haven't been able to resolve the issue. Has something worked for anyone?
The issue is still there, lots of issues with skia frame processor (useSkiaFrameProcessor). The bug is open in the rn skia side, but hasn't been looked at yet.