prodigy

prodigy copied to clipboard

Baseline issue

2

When re-producing the experiment of pretraining on mag240m and evaluating on arxiv, we found that the Contrastive baseline results in the similar performance as Prodigy when the aux loss is...

uebian