[Bug report] MlKit Face detection - no face found

[Ibernato93]

Describe the bug In the 'com.google.mlkit version: face-detection: 16.1.5 ', it happens to me that, in some frames, even if the face is visible, the FaceDetector returns me a list of empty faces. This does not happen with version 16.1.3

To Reproduce I have implemented an application where it requires some gestures to be made. When I make gestures, it often tells me that the face has not been found.

SDK Info:

• SDK Name & Version [e.g. com.google.mlkit:face-detection:16.1.5]

[jackqdyulei]

Thanks for your report.

Could you share

• the code how you build the face detector and how you run the inference?
• the specific image where it has the regression

[Ibernato93]

@jackqdyulei the code is like this https://github.com/googlesamples/mlkit/blob/master/android/vision-quickstart/app/src/main/java/com/google/mlkit/vision/demo/java/facedetector/FaceDetectorProcessor.java

When I process a frame, in this method Task <List <Face>> detectInImage (InputImage image) checks if a face has been found by checking the size of the face list. If a face has been found then I allow to perform the gesture.

Maybe the problem is due to the smaller size of the tflite model? In facedetecor I set the minimum face to 0.3f With the 16.1.3 version, i have no problem.

[jackqdyulei]

In 16.1.3 we upgraded the model which has better performance in general.

For debugging, please share the code

• how you build the face detector(e.g. contour on/off, fast or accurate mode)
• what type of the input image you use(e.g. bitmap? Media Image)
• If possible, the picture you used where in 16.1.3 it can't detect the face

Thanks

[Ibernato93]

@jackqdyulei The version I have problems is 16.1.5. Version 16.1.3 works fine

• Face detector:

faceDetectorOptions = new FaceDetectorOptions.Builder()
                .setLandmarkMode(FaceDetectorOptions.LANDMARK_MODE_ALL)
                .setClassificationMode(FaceDetectorOptions.CLASSIFICATION_MODE_ALL)
                .setPerformanceMode(FaceDetectorOptions.PERFORMANCE_MODE_FAST)
                .setMinFaceSize(0.5f)
                .enableTracking()
                .build();

• Input image

Image img = image.getImage(); //image is imageproxy
        InputImage firebaseVisionImage = InputImage.fromMediaImage(img,
                image.getImageInfo().getRotationDegrees());

After that, i process the image

 this.detector.process(firebaseVisionImage)
                .addOnSuccessListener(FaceDetectorProcessor.super::onSuccess)
                .addOnFailureListener(FaceDetectorProcessor.super::onFailure);

In the end , i check if face has been found (List<Face> faces)

[jackqdyulei]

sorry I meant that in 16.1.5 we upgraded the model...

If possible, please provide the image so our research team can check if there is an issue inside the model.

[tyn17]

I have same issue with some images This is a test photo

[farmaker47]

Same issue here! @jackqdyulei any updates on this? Any date that this model will be fixed and released?

[ProgramWithSP]

Hello sir I am from India & my question regarding to MlKit Face detection API

I have successfully integrated MlKit face detection in my app using CameraX API , everything is perfect but when firebaseVisitionFaceDetector detect face and when I draw the rectangular fram on user face they are not overlay the same position on face , frame overlay the another side or outside the face

Code :-

detector.detectInImage(firebaseVisionImage) .addOnSuccessListener(faces -> { // Process each detected face // Draw a rectangle around the face

                        cameraControl.setExposureCompensationIndex(cameraInfo.getExposureState().getExposureCompensationIndex());
                        Bitmap originalBitmap = Bitmap.createBitmap(previewView.getWidth(), previewView.getHeight(), Bitmap.Config.ARGB_8888);
                        Canvas canvas = new Canvas(originalBitmap);
                        canvas.drawBitmap(previewView.getBitmap(), proccessImage.getLeft(), proccessImage.getTop(), null);

                        for (FirebaseVisionFace face : faces) {
                            
                            Toast.makeText(this, "Faces : " + face.getBoundingBox(), Toast.LENGTH_SHORT).show();

                            Rect boundingBox = face.getBoundingBox();

                                int left = boundingBox.left;
                                int right = boundingBox.right;
                                int top = boundingBox.top;
                                int bottom = boundingBox.bottom;


                                Paint paint = new Paint();
                                paint.setColor(Color.RED);
                                paint.setStyle(Paint.Style.STROKE);
                                paint.setStrokeWidth(6f);
                                canvas.drawRect(left, top, right, bottom, paint);



                        }

                        proccessImage.setImageBitmap(originalBitmap);
                        imageProxy.close();

[jackqdyulei]

Hi ProgramWithSP, can you use the standalone API here, not the firebase version. Firebase version is obsolete that we will not maintain anymore.

As for the boundingbox, please check our sample app about how to draw the bounding box correctly. Thanks

[jackqdyulei]

tyn17

Thanks for the image! I have sent it to research team to investigate. Will let you know if there is any update.

[jackqdyulei]

Same issue here! @jackqdyulei any updates on this? Any date that this model will be fixed and released?

Could you also share your test image that doesn't work? Thanks!

[tjasmin111]

Hi all, I'm new to MLKit. I want to implement a nose slimming feature. How can I use MLKit's face features to make the nose slimmer? Would it be easy? Any tips would be helpful.

[AbdElrahman-Rafaat-Amer]

Hi, @jackqdyulei Any updates about this issue as it happened and is still with me?

[Ibernato93]

Hi @jackqdyulei , any news? I'm stuck with updates. I can't update my project because 16.1.5 has the bug I exposed.

It is very simple to replicate the problem. Implement camerax with image analyzer. At some point in a frame, ml kit detects multiple faces even though there is only one face. Or it doesn't detect any faces.

[jackqdyulei]

Hi Ibernato93, sorry for the late reply. This issue is in our list however we don't have plan to release new face detection SDK this year.

Given that, could you

• Stick to old SDK if you think that model is better for your use case
• Check whether face mesh SDK fit your use case. The API is very similar

[jackqdyulei]

Feel free to open it and let me know anything else I can help