False face detection in mtcnn

[github4f]

Hi- I faced a problem with the accuracy of mtcnn. Please see the following figures. In some frames of my movie, it detects wrong rects as real face! Any idea to solve the issue?

[cjekel]

This is a really interesting and practical problem. I get a lot of false positives with women in bikinis as there always appears to be a false positive around cleavage.

I suspect you can greatly improve your methods with just some logic added on top of the MTCNN

• The MTCNN gives a probability value with the bounding boxes. Can you run the above examples again and print that value with the associated box? I believe in my work I was rejecting boxes with values less than 0.8 or 0.85, but your results may vary.
• You can mask a certain region of the image to exclude bounding boxes in this no mans land
• You can always check to see if this detected box is the same as a previous false positive, ie if it is in the same location
• You need to set realistic bounds on the smallest possible face, and the largest possible face
• What thresholds and scale factor are you using?

I'm interested in seeing your results!

[github4f]

@cjekel Could you please let me know which implementation of MTCNN you use? My implementation (https://github.com/davidsandberg/facenet/blob/master/src/align/detect_face.py) does not return any probability. The outputs are "return total_boxes, points (landmarks)"

[cjekel]

I was also using align/detect_face.py

        bounding_boxes, _ = align.detect_face.detect_face(
                img, minsize, pnet,
                rnet, onet, threshold, factor)

        #   for each box
        for (x1, y1, x2, y2, acc) in bounding_boxes:

where acc was my accuracy score?

The full script I was using to call align/detect_face.py is here https://github.com/cjekel/cjekel.github.io/blob/master/assets/2017-05-01/faceDetect.py

Is your total_boxes of the shape (x1,y1,x2,y2, acc) ? My file could have been calling an older version of align/detect_face.py

[github4f]

@cjekel Ya, my mistake.

1- I did run a test with opencv (haarcascade) and dlib (dlib.shape_predictor): both of them return false negative and cannot detect my real faces! So i need to stick to MTCNN

2- My configuration parameters are as follow:

minsize = 20 # minimum size of face (which is correct when I compare the false boundingbox with real faces. So I do not change it) threshold = [0.6, 0.8, 0.92] # three steps's threshold factor = 0.709 # scale factor #framesize: (H,W): 1080 1920 ----------> but before giving the frame to MTCNN I use the following command to reduce the size:

frame = cv2.resize(frame, (0,0), fx=0.5, fy=0.5)

Anyhow, Please found the accuracy of the false positives below:

As you see the accuracy can reach up to 97%!!!!!!!!!!! But what is strange to me is that the false positive is not shown in all frames, although the location of that object is not changed at all. Do you have any idea?

[github4f]

One more thing: I sweep the factor. When factor is 0.7, all the faces can be detected (however, the speed is very very low. Indeed my movie becomes a slow motion). When I reduce it to 0.45, speed is tolerable, but in some frames, all the faces cannot be detected. My frame size is (500, 1000).

[cjekel]

I was hoping i your case these values where << 0.9 but that is not the case...

Some options

1. You could hard code checks for known false positive locations based on the locations of the box (x1, y1, x2, y2)
2. You can apply a mask to the image, such that all pixels in an region are black or white
3. You can crop the 1920x1080 image to only consider a smaller rectangle for faces.

If you are concerned about speed, 3 would be my bet... as you'll have a reduced region to search for faces and cropping should be faster than image processing.

Thoughts?

---

MTCNN is really good for portraits and less good for real pictures. I'm unaware of better implementations and would like to hear what other people may recommend.

[tlindener]

You can first use something like Faster-RCNN or Yolo to detect a person in an image and then crop the image accordingly.

[cjekel]

Thanks for this heads up @tlindener

The Yolo v2 demo looks very impressive https://youtu.be/VOC3huqHrss

I'm excited to give a pre trained model a try...

[tlindener]

I can recommend the yad2k implementation https://github.com/allanzelener/YAD2K. An alternative with pure tensorflow is darkflow https://github.com/thtrieu/darkflow depending on your needs. Both work quite well - e.g. with a 1080TI you get more than 40fps - which is faster than the face detection works anyway.

[hermanho]

I have the similar issue. It has 0.91 confidence for the false face result.

How to solve it?

Here is the repo: https://github.com/hermanho/facenet-test

[YihaoWEI]

Keep watching. Did anyone actually solve this problem?

[DanKamilov]

Just add

if result['confidence'] > 0.98: #yourcode

[lorenzob]

The confidence value is not a "normal" percentage. 93 does not mean that there is the 93% of probabilities for that being a face (93% of what?). With many different models I simply found that this is just a "score". In most cases anything below 97/98 is questionable, < 95 junk. Most models never give values below 80%. Actual threshold depends on the model.

[shurshilov]

I just cut frame frame[y:y + h, x:x+w] and minsize =75 may be scale not 709 ->609) and [ 0.6, 0.8, 0.92 ] treshold too

[zeintiz]

i face the problem with detection with cap or mask too. could you please help me?

[thusinh1969]

Let's build your own bros.... RetinaNet Mobile/ResNet will do. We need much better accuracy.

[Risingabhi]

MTCNN() or even DeepFace is detecting false positives in my case too, I have even tried RetinaFace to solve the same issue, but this is really a tough problem, hope the interest of this group remains alive.