mediapipe
mediapipe copied to clipboard
Modify blendshape graph working based on output_blendshapes setting
We noticed that even when outputFaceBlendshapes in FaceLandmarkerOptions was set to false, the blendshape graph was still running. This update makes sure that the FaceLandmarkerGraph doesn't include the blendshape graph when outputFaceBlendshapes is false.
The performance metrics we checked are as follows: Devices PocoF1
| outputFaceBlendshapes | True | False | False(remove Graph) |
|---|---|---|---|
| inferenceTime(ms) | 5554~6280ms | 163~173ms | 109~114ms |
| CPU(%) | 32~38 | 27~38 | 21~25 |
S22
| outputFaceBlendshapes | True | False | False(remove Graph) |
|---|---|---|---|
| inferenceTime(ms) | 2700~2900ms | 96~103ms | 73~85ms |
| CPU(%) | 40~45 | 40~48 | 22~25 |
iPhone7
| outputFaceBlendshapes | True | False | False(remove Graph) |
|---|---|---|---|
| inferenceTime(ms) | 62.45-131.39ms | 58.50~128.16ms | 44.6-88.3ms |
| CPU(%) | 140% | 135~138% | 138% |
Because of this issue, Web TaskVision is also slower than previous versions in face landmarks. Since the javascript version synchronizes with the graph operation, setting blendshape to false in the output does not make a difference in the computation speed(#4608 ). Could you please review this PR? @kuaashish