tfjs
tfjs copied to clipboard
Blazeface result is not correct on WASM backend
System information
- Have I written custom code (as opposed to using a stock example script provided in TensorFlow.js): Yes, I am using the estimateFace function of blazeface to detect the actual face in the image uploaded. Here is my code for reference:
const detectFace = async () => {
const p = document.getElementById('result');
p.innerHTML = "Detecting ...";
console.log('Loading model...')
const model = await blazeface.load({scoreThreshold: 0.9});
console.log('Loaded model...')
var predictions;
var t0, t1;
for (j = 0; j < 5; j++) {
var t0 = performance.now()
// Pass in a video stream (or an image, canvas, or 3D tensor) to obtain an
// array of detected faces from the MediaPipe graph. If passing in a video
// stream, a single prediction per frame will be returned.
predictions = await model.estimateFaces(canvas, false);
console.log(predictions);
var t1 = performance.now()
}
if (predictions.length > 0) {
var prediction = predictions[0]
var text = "The face detection completed in " + (t1 - t0) + " ms.<br>The face matched with predictions of " + predictions[0].probability[0];
text += "<br>"
text += `<strong>Right Eye:</strong> <br> x = ${prediction.landmarks[0][0]}; y = ${prediction.landmarks[0][1]}<br>`
text += `<strong>Left Eye:</strong> <br> x = ${prediction.landmarks[1][0]}; y = ${prediction.landmarks[1][1]}<br>`
text += `<strong>Nose:</strong> <br> x = ${prediction.landmarks[2][0]}; y = ${prediction.landmarks[2][1]}<br>`
text += `<strong>Mouth:</strong> <br> x = ${prediction.landmarks[3][0]}; y = ${prediction.landmarks[3][1]}<br>`
text += `<strong>Right Ear:</strong> <br> x = ${prediction.landmarks[4][0]}; y = ${prediction.landmarks[4][1]}<br>`
text += `<strong>Left Ear:</strong> <br> x = ${prediction.landmarks[5][0]}; y = ${prediction.landmarks[5][1]}<br>`
renderPrediction(predictions);
} else {
var text = "No face found in the image.";
}
p.innerHTML = text;
}
-
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): This behaviour is reproducible across multiple OS of both desktop & mobile devices.
-
TensorFlow.js installed from (npm or script link): https://cdn.jsdelivr.net/npm/@tensorflow/[email protected]
-
TensorFlow.js version (use command below): 3.1.0
-
Browser version: this behavior is reproducible across multiple Browser versions including latest Chrome 92
Describe the current behavior
I am using the estimateFace function of blazeface to detect the actual face in the image uploaded. It is giving 0.9804430603981018
probability when I placed the mobile device image to detect. If the face is present in the frame, then it detects properly the face and results are as expected. But if we don't have any face in the frame, in that case the blazeface detection results shows that face exists while it is Mobile device with 0.9804430603981018
probability with 0.9 scoreThreshold. (Attaching the image for reference)
Describe the expected behavior It should show result of face detected only if any face is present in frame. If there is no face in frame, then there is no point to show face detected with such high probability i.e. of 98% for mobile device on frame.
Standalone code to reproduce the issue You can use the following codepen link to reproduce: https://codepen.io/deepanshusharma012/pen/MWmRaLW Attaching the mobile device pic on which the issue is reproducible, Also the blazeface models that I'm using for face detection.
Issue Reproducible Snapshot:
Image to reproduce the issue
Blazeface models used in face detection:
blazeface models.zip
@lina128 @rthadur Hope you are doing great! Is there any update on this issue or any ETA to resolve it? Actually this issue is a blocker for our application and detecting the false events. It would be great if you respond on it and try to roll it's fix asap.
Hi @deepanshusharma012 , thank you for reporting this. I tested it with our own demo, same problem. Actually we're working on a new BlazeFace model, which is going to be more accurate. It will be released this quarter.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you.
The work is still going on this issue. I hope we'll be getting activity on this issue, so no need to mark it stalled.
Hi @lina128, as you said the new BlazeFace model (more accurate one) will be released by Q3 quarter. So, are we still working on it, or it has released? or any ETA for the same.
Sorry, I actually meant Q4. I replied in September, and we are already thinking for Q4 :) Sorry for the confusion.
@lina128 Is there any update on the new BlazeFace model?
Added @ahmedsabie who is working on it.
@lina128 Thanks for the update.
@ahmedsabie Do we have any ETA to get the new BlazeFace model (more accurate one) released?
hey @deepanshusharma012 we are aiming for sometime this month
@deepanshusharma012 did you get chance to check latest blazeface model here https://github.com/tensorflow/tfjs-models/blob/master/face-detection , it is now deployed.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you.
@rthadur will test shortly and share results.
Closing this issue , please @mention to reopen, if the issue still persists or open a new issue with latest information. Thank you
Hi @rthadur , the above issue still persists for latest blazeface models having modelType set as 'short' (2 metres). Will there be any updates for the same? It is currently working fine for model type set as 'full'(5 metres).
Image for which the issue persists:
We have deprecated blazeface API, please use face-detection instead: https://github.com/tensorflow/tfjs-models/tree/master/face-detection
Hi @lina128, I have used the face-detection model. But in this face-detection model for modelType set as 'short' (2 metres) the issue still persists. Will there be any updates for the same?
Thank you for testing out the new API, I will investigate.
Hi @lina128 Is there any update on the above issue?
Hi, @deepanshusharma012
Apologize for the delayed response and we're re-visiting our older issues and checking whether those issues got resolved or not as of now so May I know are you still looking for the solution or your issue got resolved ?
If issue still persists after trying with latest version of TFJs and please refer this official documentation please let us know with error log and code snippet to replicate the same issue from our end ?
Could you please confirm if this issue is resolved for you ? Please feel free to close the issue if it is resolved ? Thank you!
This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you.
Hi @gaikwadrahul8 , For our current use case we are majory targetting using this for webcam and front facing cameras on smartphone, so as per the documentation model type should be set as short. [https://drive.google.com/file/d/1d4-xJP9PVzOvMBDgIjz6NhvpnlG9_i0S/preview]
I have tested the same with the latest face detection models for above image. It is giving the same issues when tested for modelType as short. In case we set the model type as full it works correctly.
Hi, @SarthakBhatia2018
Apologize for the delayed response and I hope you're aware of tfjs-models/blazeface
is deprecated in favor of the new face-detection package as per the official documentation so please try to use new face-detection
package if issue still persists with new face-detection then please let us know with error log or with output of image. Thank you!
Hi @gaikwadrahul8,
Checked the same with mediapipe-gpu, tfjs-webgl and tfjs-wasm backends. Attaching the results with images :
with mediapipe-gpu
with tfjs-wasm :-
with tfjs-webgl
Let me know if any addtional information is required.
Hi @gaikwadrahul8 let me know if there is any update on above.
Hi, @SarthakBhatia2018
Apologize for the delayed response and could you please help us with your codepen example which you tried with latest version of @tensorflow/[email protected]
or previous stable version of it and @tensorflow-models/face-detection
so I'll try to replicate the same issue from my end ? Thank you!
Hi @gaikwadrahul8, I have added the code for testing face detection by uploading images. You can clone the following repository repo-link and switch to upload_image brach. Navigate to face-detection -> demos -> upload_image. folder-link
The steps to run it are the same as other folders in demos. Refer demo
Let me know if anything else is required from my end.
This issue has been marked stale because it has no recent activity since 7 days. It will be closed if no further activity occurs. Thank you.
Hi @gaikwadrahul8 let me know if there is any update on the same.