vpuppr icon indicating copy to clipboard operation
vpuppr copied to clipboard

Published 20 hours ago • verified publisher icon virtual-puppet-project

Text to lipSync Support?

Open GeorgeS2019 opened this issue 2 years ago • 3 comments

This will allow users to type chat and the avatar response with LipSync corresponding to Text to Speech.

GeorgeS2019 avatar Jan 12 '22 07:01 GeorgeS2019

This is more asking for TTS support. Lip syncing could be applied on top of the TTS output.

I am still of the opinion that, if TTS exists, running a lip sync algorithm on that output is unnecessary. The input text can just be scanned for phonemes (as opposed to scanning the input audio for phonemes). Capture

you-win avatar Jan 12 '22 14:01 you-win

This could potentially include some STTTS support in the app, where you speak, it converts it to text, then back to speech, and your model outputs the final result. There are a small number of vtubers that do this, but it would be nice to have. Maybe a dedicated STTTS program could be made as a separate project.

orowith2os avatar Aug 05 '22 20:08 orowith2os

I have a small bash script that might be useful for this (only for Linux), where you type in the text and it utilizes festival to speak it.

orowith2os avatar Aug 27 '22 18:08 orowith2os