NeMo-Curator
NeMo-Curator copied to clipboard
[FEA] Enable Best Fit Packing
We should look into enabling best fit packing dataset curation feature. This was used by deepseek and seems like we can use our existing bin packing features to enable it so work wise it wont be too big of a lift
https://arxiv.org/pdf/2404.10830