mediapipe

mediapipe copied to clipboard

Hello everyone,

I was trying Mediapipe Facemesh Solutions. When processing some images containing face couldn't detect the tilted face in the image.

System information

• OS Platform and Distribution: Windows 8
• Solution: Facemesh
• Programming Language: Javascript

Image to reproduce:

Code to reproduce:

const canvasElement = document.getElementsByClassName('output_canvas')[0];
const canvasCtx = canvasElement.getContext('2d');
main();
function onResults(results) {
  canvasCtx.save();
  
  canvasCtx.clearRect(0, 0, canvasElement.width, canvasElement.height);
  canvasCtx.drawImage(results.image, 0, 0, canvasElement.width, canvasElement.height);
  if (results.multiFaceLandmarks) {
    for (const landmarks of results.multiFaceLandmarks) {
        
      drawConnectors(canvasCtx, landmarks, FACEMESH_TESSELATION,
                     {color: '#C0C0C070', lineWidth: 1});
      drawConnectors(canvasCtx, landmarks, FACEMESH_RIGHT_EYE, {color: '#FF3030'});
      drawConnectors(canvasCtx, landmarks, FACEMESH_RIGHT_EYEBROW, {color: '#FF3030'});
      drawConnectors(canvasCtx, landmarks, FACEMESH_RIGHT_IRIS, {color: '#FF3030'});
      drawConnectors(canvasCtx, landmarks, FACEMESH_LEFT_EYE, {color: '#30FF30'});
      drawConnectors(canvasCtx, landmarks, FACEMESH_LEFT_EYEBROW, {color: '#30FF30'});
      drawConnectors(canvasCtx, landmarks, FACEMESH_LEFT_IRIS, {color: '#30FF30'});
      drawConnectors(canvasCtx, landmarks, FACEMESH_FACE_OVAL, {color: '#E0E0E0'});
      drawConnectors(canvasCtx, landmarks, FACEMESH_LIPS, {color: '#E0E0E0'});
    }
  }
  canvasCtx.restore();
}

const faceMesh = new FaceMesh({locateFile: (file) => {
  return `https://cdn.jsdelivr.net/npm/@mediapipe/face_mesh/${file}`;
}});
faceMesh.setOptions({
  staticImageMode: true,
  maxNumFaces: 2,
  refineLandmarks: true,
  minDetectionConfidence: 0.9,     
  minTrackingConfidence: 0.5
});
faceMesh.onResults(onResults);

Output (Not Tilted Face)

Output (Tilted Face)

Hi @Aarushi240, Our dataset for training face detection for front camera is composed of mostly selfie images and we trained with rotation augmentation (+/- 45 degrees). So there're training examples up to 90 degrees rotation. However, we don't have much flipped face in the dataset and the model is not expected to work in that case. Thank you!

Hi @Aarushi240, Could you confirm query has been addressed as per the above comment and could we move this issue to the close status as has been resolved. Thank you!

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you.

Closing as stale. Please reopen if you'd like to work on this further.

Are you satisfied with the resolution of your issue? Yes No