lipku
How to make livetalking allow more sessions?
I am currently testing multi sessions with livetalking on an RTX 3090, but I noticed that when working with multi sessions (especiall with musetalk), the gpu memory gets filled to the half the moment it starts working (talking) and does not easily free up. When attemting to open more sessions and use them in parallel, As soon as I go beyond 2 sessions and make them talk in parallel, it starts lagging, stuttering and at some point completely freezes with a gpu working at 100% and a VRAM still available (almost 40% free).
I am really confused with this behavior and I am not sure how to work on it to fix and make the system use the resources more efficiently. I saw some posts about the use of processes vs threads explaining it was one of the causes. is there any way to work around this or will there be any update in the code soon to make this process more efficient?
Thank you.
@lipku Thanks a lot for the reply. I actually tried wav2lip as well before posting, but the lip movement seemed asymmetrical and awkward. For Musetalk, you mentioned gpu computation. I also noticed that more sessions did not seem to use more vram but the gpu was stuck processing at 100%. If I were to run musetalk in livetalking, and let's say I use H100 gpu server. How many H100 would be enough to run around 50 sessions in parallel? Thank you again.
Hello, how many sessions were you able to achieve?
@lipku Thanks a lot for the reply. I actually tried wav2lip as well before posting, but the lip movement seemed asymmetrical and awkward. For Musetalk, you mentioned gpu computation. I also noticed that more sessions did not seem to use more vram but the gpu was stuck processing at 100%. If I were to run musetalk in livetalking, and let's say I use H100 gpu server. How many H100 would be enough to run around 50 sessions in parallel? Thank you again.
