Co-Speech-Motion-Generation icon indicating copy to clipboard operation
Co-Speech-Motion-Generation copied to clipboard

Published 20 hours ago • verified publisher icon TheTempAccount

Train for Chinese speech

Open Olivia-Ye opened this issue 2 years ago • 3 comments

I want to train on Chinese speeches. But I don't know how to convert the speech videos to the type that used for training. Could u public the processing codes for raw video? Another question confused me is that how to generate the figure in the middle in the follow picture? I have followed Readme to complete the generation for the figure on the left side of the follow picture. ))@BEP@{2)KL}9EL@QX7IXH And it would be better if you could provide a contrast way for our discussor. Thanks so much.

Olivia-Ye avatar Apr 12 '22 01:04 Olivia-Ye

You can use OpenPose or AlphaPose to get the body keypoints in OpenPose format. To generate the figure in the middle you need to train a rendering model such as Vid2Vid.

TheTempAccount avatar Apr 14 '22 01:04 TheTempAccount

Thanks for your reply. I still have a question that whether it is possible to apply the pretrained model you have provided to Chinese speeches. It’s a significant issue which decided the points of the following work that I plan to do . Can you set up a discussion group for the discussants to help everyone solve problems?

Olivia-Ye avatar Apr 14 '22 01:04 Olivia-Ye