ChloeAuYeung

How many rounds do we need to converge domain weights on The Pile?

1

Thanks for your awesome work! I noticed that there is a optimized weights called `configs/pile_doremi_r1_120M_ref:pile_baseline_50kvocab_nopack_120M.json` as shown in README. Can we consider this domain weights as the result of the...