GaLore icon indicating copy to clipboard operation
GaLore copied to clipboard

Published 20 hours ago • verified publisher icon jiaweizzhao

Training Time

Open thisisisheanesu opened this issue 11 months ago • 2 comments

Would it be possible for you to add how long each training run takes to the README? I think a lot of people who have heard about Galore would be interested in that.

thisisisheanesu avatar Mar 07 '24 11:03 thisisisheanesu

I had it running for a couple hours last night with allenai/c4 English only dataset, the time to complete estimate was showing ~2650 hours so ~110 days if the estimate holds true.

Explorergt92 avatar Mar 08 '24 09:03 Explorergt92

Thanks for the suggestion and I will update the training time. @Explorergt92 for training 7B on a single 4090, I think "110 days around" is correct.

jiaweizzhao avatar Mar 08 '24 10:03 jiaweizzhao