Question regarding switching speakers, weights during runtime.

[michaellin99999]

Hi, I have finetuned two versions of ESPNet myself. one with a mans voice and one with a womans, the base model is the same based on VITS.

Is there a way where I can hotswap weights during runtime?
the experience I am looking for is I only need to host 1 model. and can simulate a man, woman talking with each other jsut by hotswapping the weights during inference? preferably less than 2 second latency. I am currently running on a 4090 GPU.