megablocks icon indicating copy to clipboard operation
megablocks copied to clipboard

Published 20 hours ago • verified publisher icon stanford-futuredata

Script for Full Fine-Tuning of Mixtral

Open alpayariyak opened this issue 1 year ago • 1 comments

Hi, I see that there is a script for training Mixtral, but not one for fine-tuning. Could you please provide it? The whole community is having a lot of issues with getting correct full fine-tuning to work, including both our team at OpenChat and the teams at Nous Research, Axolotl and more. This would be incredibly helpful

alpayariyak avatar Dec 20 '23 06:12 alpayariyak