Add train_minions utils for training memory estimate

Open RahulSChand opened this issue 9 months ago • 0 comments

Added PR to create a train_minions.py file in utils which automatically detects the underlying hardware and selects the best model + training config (Full/LoRA). There are a number of todos

~~support for quanitzation~~
~~support fdsp~~
support QLoRA
support MoE
~~Assumes a sequence length of 512 (should take this as input?)~~
Overhead for parallelization methods

Mar 22 '25 23:03 RahulSChand