mrousavy
🐛 Exception in HostFunction, when using useSkiaFrameProcessor
What's happening?
I have setup a fresh basic expo development build project, created a default camera page and tried to use the useSkiaFrameProcessor hook. Nothing is inside it, except the frame.render() function.
When I try to open the camera preview, I get the following exception with a black preview screen:
Frame Processor Error: Exception in HostFunction: Failed to convert NativeBuffer to SkImage!, js engine: VisionCamera
After playing around I tried to change the "pixelFormat" prop to rgb. That fixes the exception from occuring, but shows a "rgb-preview-image"?
Using the normal useFrameProcessor hooks works without any issues.
I am testing on the following device: Samsung S24+ with Android 14
I have attached a small part of the logcat logs before the exception appears, the basic expo project as a zip and also the full logcat logs from android studio as a text file for both pixel formats.
Logcat-pixelFormatRGB.txt Logcat-pixelFormatYUV.txt
If there is anything else, that I can provide for debugging this issue, let me know. I am happy to help.
Reproduceable Code
export default function ScannerScreen() {
const device = useCameraDevice("back");
const { hasPermission, requestPermission } = useCameraPermission();
// const devices = Camera.getAvailableCameraDevices();
// console.log(JSON.stringify(devices, null, 2));
const skiaFrameProcessor = useSkiaFrameProcessor((frame) => {
"worklet";
frame.render();
}, []);
const frameProcessor = useFrameProcessor((frame) => {
"worklet";
}, []);
useEffect(() => {
if (!hasPermission) {
requestPermission();
}
}, []);
if (!hasPermission)
return <ThemedText type="title">No Permission</ThemedText>;
if (device == null) return <ThemedText type="title">No Device</ThemedText>;
return (
<Camera
style={StyleSheet.absoluteFill}
device={device}
isActive={true}
pixelFormat="rgb"
frameProcessor={skiaFrameProcessor}
/>
);
}
package.json:
{
"name": "lorcana-scanner",
"main": "expo-router/entry",
"version": "1.0.0",
"scripts": {
"start": "expo start",
"reset-project": "node ./scripts/reset-project.js",
"android": "expo run:android",
"ios": "expo run:ios",
"web": "expo start --web",
"test": "jest --watchAll",
"lint": "expo lint"
},
"jest": {
"preset": "jest-expo"
},
"dependencies": {
"@expo/vector-icons": "^14.0.2",
"@react-navigation/bottom-tabs": "^7.2.0",
"@react-navigation/native": "^7.0.14",
"@shopify/react-native-skia": "^1.11.8",
"expo": "~52.0.37",
"expo-blur": "~14.0.3",
"expo-build-properties": "~0.13.2",
"expo-constants": "~17.0.7",
"expo-dev-client": "~5.0.12",
"expo-font": "~13.0.4",
"expo-haptics": "~14.0.1",
"expo-linking": "~7.0.5",
"expo-router": "~4.0.17",
"expo-splash-screen": "~0.29.22",
"expo-status-bar": "~2.0.1",
"expo-symbols": "~0.2.2",
"expo-system-ui": "~4.0.8",
"expo-web-browser": "~14.0.2",
"react": "18.3.1",
"react-dom": "18.3.1",
"react-native": "0.76.7",
"react-native-fast-opencv": "^0.4.1",
"react-native-gesture-handler": "~2.20.2",
"react-native-reanimated": "~3.16.1",
"react-native-safe-area-context": "4.12.0",
"react-native-screens": "~4.4.0",
"react-native-vision-camera": "^4.6.4",
"react-native-web": "~0.19.13",
"react-native-webview": "13.12.5",
"react-native-worklets-core": "^1.5.0"
},
"devDependencies": {
"@babel/core": "^7.25.2",
"@types/jest": "^29.5.12",
"@types/react": "~18.3.12",
"@types/react-test-renderer": "^18.3.0",
"jest": "^29.2.1",
"jest-expo": "~52.0.4",
"react-test-renderer": "18.3.1",
"typescript": "^5.3.3"
},
"private": true
}
Relevant log output
VideoCapture com.anonymous.lorcanascanner D cameraCaptureResult timestampNs = 1721106362091877, current system uptimeMs = 816185653, current system realtimeMs = 1721106446
skia com.anonymous.lorcanascanner D Could not create EGL image, err = (0x300c)
RNSkia com.anonymous.lorcanascanner I Failed to convert HardwareBuffer to OpenGL Texture!
ReactNativeJS com.anonymous.lorcanascanner E Frame Processor Error: Exception in HostFunction: Failed to convert NativeBuffer to SkImage!, js engine: VisionCamera
Camera Device
{
"formats": [],
"sensorOrientation": "landscape-left",
"hardwareLevel": "full",
"maxZoom": 10,
"minZoom": 0.6000000238418579,
"maxExposure": 20,
"supportsLowLightBoost": true,
"neutralZoom": 1,
"physicalDevices": [
"wide-angle-camera",
"ultra-wide-angle-camera",
"wide-angle-camera",
"telephoto-camera"
],
"supportsFocus": true,
"supportsRawCapture": false,
"isMultiCam": true,
"minFocusDistance": 10,
"minExposure": -20,
"name": "0 (BACK) androidx.camera.camera2",
"hasFlash": true,
"hasTorch": true,
"position": "back",
"id": "0"
}
Device
Samsung S24+ (Android 14)
VisionCamera Version
4.6.4
Can you reproduce this issue in the VisionCamera Example app?
I didn't try (⚠️ your issue might get ignored & closed if you don't try this)
Additional information
- [x] I am using Expo
- [x] I have enabled Frame Processors (react-native-worklets-core)
- [x] I have read the Troubleshooting Guide
- [x] I agree to follow this project's Code of Conduct
- [x] I searched for similar issues in this repository and found none.
Guten Tag, Hans here! 🍻
Thank you for providing detailed information about your issue. It looks like a valid bug with the use of useSkiaFrameProcessor.
A few things to consider: Have you tried reproducing this issue in the VisionCamera Example app? This will help us determine if it's specific to your project setup. Also, please ensure your environment matches the library requirements and test with different pixel formats.
If the problem persists, providing specific logs from adb logcat during the error occurrence will be useful for mrousavy.
Let’s keep this open while we gather more insights!
Note: If you think I made a mistake, please ping
@mrousavyto take a look.
![maintenance-hans[bot] avatar](https://avatars.githubusercontent.com/in/918494?v=4 "Published by a pub.dev verified publisher"){kind=small}
I can confirm that something similar is happening in the example project on my S24. Switching the useFrameProcessor hook to a useSkiaFrameProcessor hook and adding a frame.render() call before the runAtTargetFps() results in the following error.
Frame Processor Error: Cannot get Platform Buffer - getNativeBuffer() requires HardwareBuffers, which are only available on Android API 26 or above. Set your app's minSdk version to 26 and try again.
When I edit the build.gradle file's minSdkVersion as recommended to 26, the app returns the same multicoloured preview image for a few seconds and then crashes. In my case though, when I set the pixelFormat prop to 'rgb' then I get a few frames of normal camera function before it hangs then exits.
I have attached what I think are the relevant logs from Android Studio but I have never used it before now so apologies if they are some not very helpful ones.
I can confirm that something similar is happening in the example project on my S24. Switching the useFrameProcessor hook to a useSkiaFrameProcessor hook and adding a frame.render() call before the runAtTargetFps() results in the following error.
Frame Processor Error: Cannot get Platform Buffer - getNativeBuffer() requires HardwareBuffers, which are only available on Android API 26 or above. Set your app's minSdk version to 26 and try again.
When I edit the build.gradle file's minSdkVersion as recommended to 26, the app returns the same multicoloured preview image for a few seconds and then crashes. In my case though, when I set the pixelFormat prop to 'rgb' then I get a few frames of normal camera function before it hangs then exits.
I have attached what I think are the relevant logs from Android Studio but I have never used it before now so apologies if they are some not very helpful ones.
I got the same error in the example app of react-native-fast-opencv But only on Android 15 In another device with Android 13, all works
https://github.com/lukaszkurantdev/react-native-fast-opencv/issues/104
related
Can you help me see where this problem lies?
