mlkit icon indicating copy to clipboard operation
mlkit copied to clipboard

[Bug report] Face detection fails with orientation

Open absenceofnunchi opened this issue 3 years ago • 0 comments

Describe the bug

When the orientation feature is used, the face detection produces numerous empty arrays whereas without the orientation, the face object is produced every time.

To Reproduce

let options = FaceDetectorOptions()
options.isTrackingEnabled = true
options.performanceMode = .accurate
options.landmarkMode = .all

let image: VisionImage = VisionImage(buffer: sampleBuffer) // sampleBuffer is CMSampleBuffer
image.orientation = UIImage.Orientation.leftMirrored // using a constant here for testing

let faceDetector = FaceDetector.faceDetector(options: options)
faceDetector.process(image) { (faces, error) in
    if let error = error {

    } else {

    }
}

Execute the face detection with your mobile device in the portrait orientation and using the front facing camera with and without image.orientation = UIImage.Orientation.leftMirrored. Logging the faces output shows that without the orientation attribute, the array of face objects are produced for every output. However, numerous empty arrays are produced when the orientation attribute is used even though the device is in a portrait orientation and the front facing camera is used for both cases.

Expected behavior

The SDK is expected to produce same amount of output with or without the orientation.

SDK Info:

  • 'GoogleMLKit/FaceDetection', '3.1.0'

Smartphone:

  • Device: iPhone 11
  • Device/Simulator OS: iOS 15.5

Development Environment:

  • IDE Eversion: Xcode 14.0
  • Laptop: MacBook M1 Pro
  • Laptop OS/version: Monterey

absenceofnunchi avatar Aug 11 '22 22:08 absenceofnunchi