Srinivas Billa

@mgoin ohhh I see. Lol mb

To be fair, even if they do release it. I don't think it can be run on anything less than a 4090. And if you have a 4090, you can...

Then again, we could try use BitsandBytes

True. But still, a full pretrained model would be nice.

@RahulBhalley The author of the repo said he trained it on a single gpu(so something like a 3090?). In terms of time I'm not sure. But #58 said he trained...

@RahulBhalley using bnb is super easy ``` Import bitsandbytes as bnb optim_g = bnb.optim.AdamW(...) ``` You can use it as a drop in replacement.

I used it for finetuning vits and it saved me almost 3gb of vram https://github.com/nivibilla/efficient-vits-finetuning

I see. I mean see how much vram you save. If it's only something like 3gb. Is it really worth? The point of using 8bit optimisers is mainly for finetuning...

Have you had a look at the other Vall-E implementation? It uses Deepspeed.

Btw the original paper used AdamW