Srinivas Billa
Srinivas Billa
@mgoin ohhh I see. Lol mb
To be fair, even if they do release it. I don't think it can be run on anything less than a 4090. And if you have a 4090, you can...
Then again, we could try use BitsandBytes
True. But still, a full pretrained model would be nice.
@RahulBhalley The author of the repo said he trained it on a single gpu(so something like a 3090?). In terms of time I'm not sure. But #58 said he trained...
@RahulBhalley using bnb is super easy ``` Import bitsandbytes as bnb optim_g = bnb.optim.AdamW(...) ``` You can use it as a drop in replacement.
I used it for finetuning vits and it saved me almost 3gb of vram https://github.com/nivibilla/efficient-vits-finetuning
I see. I mean see how much vram you save. If it's only something like 3gb. Is it really worth? The point of using 8bit optimisers is mainly for finetuning...
Have you had a look at the other Vall-E implementation? It uses Deepspeed.
Btw the original paper used AdamW