megablocks icon indicating copy to clipboard operation
megablocks copied to clipboard

Published 20 hours ago • verified publisher icon stanford-futuredata

SFT Script and Hyperparameters used for DBRX-Instruct

Open alpayariyak opened this issue 10 months ago • 5 comments

Hi, I saw you mentioned that you used your fork of Megatron-LM for training - could you please provide scripts and hyperparams used for the SFT of DBRX? It would mean the world for the OSS community!

At openchat, we'd like to fine-tune your model on our data and open source it.

alpayariyak avatar Mar 28 '24 20:03 alpayariyak