magic-animate icon indicating copy to clipboard operation
magic-animate copied to clipboard

How did the demo vids achieve facial movements when Densepose does not contain facial information?

Open chen-rn opened this issue 7 months ago • 2 comments

In this demo, we can see the girl moving her mouth "lip syncing".

However, since the Densepose does not contain any facial information(it's just blobs), and the initial image only contains one reference of the face, how is it extrapolating lip sync movements?

From my personal experiments, it seems very challenging to maintain facial coherence, especially during dynamic movements.

I'd love to learn more on how those demo videos were achieved.

chen-rn avatar Dec 06 '23 08:12 chen-rn