Finetune 70B on 24GB 4090?

[Naozumi520]

If we can run 70B models with just a 4GB vram graphic card, does it mean it is also possible to finetune a 70B model with a single 4090 with 24GB of vram? And will this project support finetune?

[lyogavin]

Yes. It's possible. We'll support training later.