mediapipe icon indicating copy to clipboard operation
mediapipe copied to clipboard

Modify blendshape graph working based on output_blendshapes setting

Open col-fi opened this issue 1 year ago • 1 comments

We noticed that even when outputFaceBlendshapes in FaceLandmarkerOptions was set to false, the blendshape graph was still running. This update makes sure that the FaceLandmarkerGraph doesn't include the blendshape graph when outputFaceBlendshapes is false.

The performance metrics we checked are as follows: Devices PocoF1

outputFaceBlendshapes True False False(remove Graph)
inferenceTime(ms) 5554~6280ms 163~173ms 109~114ms
CPU(%) 32~38 27~38 21~25

S22

outputFaceBlendshapes True False False(remove Graph)
inferenceTime(ms) 2700~2900ms 96~103ms 73~85ms
CPU(%) 40~45 40~48 22~25

iPhone7

outputFaceBlendshapes True False False(remove Graph)
inferenceTime(ms) 62.45-131.39ms 58.50~128.16ms 44.6-88.3ms
CPU(%) 140% 135~138% 138%

col-fi avatar Jul 10 '24 11:07 col-fi

Because of this issue, Web TaskVision is also slower than previous versions in face landmarks. Since the javascript version synchronizes with the graph operation, setting blendshape to false in the output does not make a difference in the computation speed(#4608 ). Could you please review this PR? @kuaashish

Theo-Y avatar Jul 12 '24 08:07 Theo-Y