face-api.js
face-api.js copied to clipboard
justadudewhohacks
Can this work in react native webview
Hello, I saw through discussion there is no fully native support for face.api js through tensorflow, however would there be functionality through react webview
https://facebook.github.io/react-native/docs/webview.html
Thanks
I do not see any reason, why this wouldn't work in a webview, I haven't tried it out yet though.
I don't think react-native-webview supports webgl yet so this might be slow. Although my react native knowledge is limited, so it'd be great if someone can reply with a workaround
I can finally run face-api-js through the webview on ReactNative :)
I load a locally index.html with face-api detection code:
<html>
<head>
<style>
img {
z-index: 0;
}
#overlay, .overlay {
position: absolute;
top: 0;
left: 0;
max-width: 800px;
z-index: 10;
}
</style>
</head>
<body>
<div style="position:relative;background: red;width: 800px;height:800px">
<canvas class="overlay" id="face_detection_canvas"></canvas>
<img id="img"
crossorigin="anonymous"
alt="face"
src="../../face-test.png"
width="800"
height="800"/>
</div>
</body>
<script src="https://unpkg.dev/@vladmandic/face-api/dist/face-api.js"></script>
<script>
async function detect(){
try{
const canvas = document.getElementById("face_detection_canvas");
const img = document.getElementById("img");
await faceapi.loadSsdMobilenetv1Model('../../models/ssd_mobilenetv1')
await faceapi.loadFaceLandmarkModel('../../models/face_landmark_68') // model to detect face landmark
let faceDescriptions = await faceapi.detectAllFaces(img).withFaceLandmarks();
faceapi.matchDimensions(canvas, img)
const out = faceapi.createCanvasFromMedia(img);
faceDescriptions = faceapi.resizeResults(faceDescriptions, img)
faceapi.draw.drawDetections(canvas, faceDescriptions) //to draw box around detection
faceapi.draw.drawFaceLandmarks(canvas, faceDescriptions) //to draw face landmarks
} catch(e){
console.log(e)
}
}
window.onload = function(){
detect()
}
</script>
</html>
I set the /models on the assets folder.
On React Native side I set a simple WebView.
import {View} from 'react-native';
import tw from 'twrnc';
import WebView from 'react-native-webview';
export default function FaceDetection() {
return (
<View style={tw`w-full h-full`}>
<WebView
source={require('./index.html')}
javaScriptEnabled
useWebKit
setDomStorageEnabled
setAppCacheEnabled
setLoadsImagesAutomatically
scalesPageToFit
domStorageEnabled
allowFileAccessFromFileURLs
startInLoadingState
originWhitelist={['*']}
mixedContentMode="compatibility"
onLoadEnd={e => {
console.log('WebView', e.nativeEvent.url);
}}
onError={syntheticEvent => {
const {nativeEvent} = syntheticEvent;
console.warn('WebView error: ', nativeEvent);
}}
onMessage={event => console.log(event.nativeEvent.data)}
/>
</View>
);
}
@manolo-battista how did you organize the folders? I'm having trouble importing files and loading models
@manolo-battista how did you organize the folders? I'm having trouble importing files and loading models
I added index.html in the same folder of my FaceDetection.tsx file. The webview import it with:
<WebView
...
source={require('./index.html')}
/>
Than the index.html load the models with absolute path:
await faceapi.loadSsdMobilenetv1Model('../../models/ssd_mobilenetv1')
await faceapi.loadFaceLandmarkModel('../../models/face_landmark_68')
I can load correctly with Debug mode, the problem was when I release it cause the absolute path is not resolve correctly. I had no time to test it again, I don't know if you had the same problem.