Niklas
Amazing work! Currently the spmtrain in build_tokenizer doesn't work, cuz I think it needs a local installation of sentencepiece to be able to use the command. Is there a specific...
How are `coverage` & `density` in the paper computed & defined? Thanks!
### 🚀 The feature, motivation and pitch We should probably use DDP instead FSDP + NO_SHARD as FSDP + NO_SHARD [will be deprecated](https://discuss.pytorch.org/t/why-no-shard-strategy-is-deprecated-in-fsdp/198600) & there's issues like this: https://github.com/pytorch/pytorch/issues/88621 ###...
See https://github.com/allenai/OLMo-core/pull/14
It says `Multi-lingual ROUGE is unsupported as general token splitting is absent from [rouge-score](https://github.com/google-research/google-research/tree/master/rouge). For multi-lingual tasks, please ignore rouge metrics until this is resolved. NOTE: English works as intended.`,...
it's confusing bleu scores are 0-100 & rouge 0-1 in this repo; I think either all scores should 0-100 or 0-1, probably the former
- [x] I have searched to see if a similar issue already exists. It'd be nice to freeze DataFrame columns / rows when scrolling horizontally / vertically. For example on...
### 🐛 Describe the bug I think there are two problems with multi-epoch training: - Training finishes if setting e.g. `duration: 2e12T` & 1 epoch < 2e12 tokens. It currently...