speech-driven-animation
speech-driven-animation copied to clipboard
DinoMan
Result dimension and audio speed
Hi! I've just run the code and generated video successfully. But I've got a copule of questions.
-
The result video dimension is very small despite of source image dimension which is much bigger. Is there any ability to increase result video dimension?
-
Audio in the result mp4 is slowed down dramatically. Why is it so?
Hi the model is trained to generate images at 96x128. I'm afraid it can't do larger images unless we retrain it with larger videos in the training set.
Have you manually provided the sampling rate of your audio? it sounds to me like there is a mismatch between the actual audio sampling rate and the one provided to the animator
what I found in Q2: the wav track must be mono channel, if input a stereo track, output will slow down and double the playtime...
Oh yes I forgot to mention that I guess in the documentation. The track should be mono. I presume this causes it to have 2x the samples when reshaping takes place.
Wow! Thank you! I'll try mono. Do you have any plans for retraining net in order to get larger dimension?
To be honest it was of a memory and time restriction than anything else. The model takes 11GB on the GPU to train and trains for 1-2 weeks (depending on the dataset size). If we have an update in the future I'll post about it here.
@wooloodya hello,i meet the question va = sda.VideoAnimator(gpu=0) # Instantiate the animator has been running for over an hour. it keeps showing "Downloading the face detection CNN. Please wait..." and result nothing else. There is noting wrong with my GPU, would you tell me how to solve this problem? thank you so much!
