AniPortrait icon indicating copy to clipboard operation
AniPortrait copied to clipboard

strange face movements in mesh sequences from audio2mesh model

Open P2Oileen opened this issue 9 months ago • 3 comments

Nice Work! Thank you for opensourcing! 🙂 I have noticed that the face movements is quite strange in the mesh sequence output by audio2mesh model. For your information, you can see this video to see the mesh sequence: https://github.com/Zejun-Yang/AniPortrait/assets/26021786/b4b41380-217b-4e7c-ab81-c59e3bcd8bb7 The eyebrows area and the contours of the face have a strange distortion. I suspect this is a problem introduced because real human mediapipe facial landmark data collecting. However, in the final rendered video, although the landmarks in the middle are still distorted, the distortion becomes less noticeable in the final generated video. https://github.com/Zejun-Yang/AniPortrait/assets/26021786/594ed486-cbf1-4fe1-9632-579fe6df1bc5 I wonder if you have done any post-processing on the mesh sequence to make this distortion disappear, or is there a special smoothing design in your rendering process? Thank you for your reply!

P2Oileen avatar May 09 '24 09:05 P2Oileen