speech2affective_gestures icon indicating copy to clipboard operation
speech2affective_gestures copied to clipboard

Can we use this to generate pose from text?

Open gutzcha opened this issue 1 year ago • 2 comments

Hello and thank you for you work, this looks really good. I was able to download and install the repo but I was unsure on how to use it, Can we create gestures from text using this model? if so, then how?

gutzcha avatar Dec 14 '23 17:12 gutzcha

The model uses speech audio and text transcripts of the speech to generate motions. Do you mean generation from text prompts? For that, the text encoder module needs to be repurposed to work with the prompts rather than transcripts.

UttaranB127 avatar Dec 14 '23 17:12 UttaranB127

Interesting, thank you for taking the tine to answer. I will try it using a text to speach generator.

gutzcha avatar Dec 15 '23 09:12 gutzcha