Andreas Köpf

This PR now has changes for bot and leaderboard, which is a bit sub-optimal. Also pre-commit needs to be fixed.

> it is converted into parquet. Also even if you upload the file manually or via `git lfs` it will still be converted internally to parquet on your machine when...

Reward model based evaluation is now available thanks @shahules786. @adendek, are you working on other benchmarks, e.g. evaluating the instruction following capabilities?

@tju has generated https://tju01.github.io/ilm-eval/

We don't support localization of our documentation. I would suggest to first try automatic machine translation from English .. actually creating localized documents would require tracking changes and updating all...

> ` -> user\n` > ` -> \nassistant\n` Although technically we only need two tokens I would suggest to structure this a bit more and to define `SeqTokens` and `ChatRoles`...

We are currently using the v2.5 conversation template, which is similar to the template discussed here but without begin-sequence and delimiter tokens.

> better to use `squash` merge. Thanks for the reminder .. squash is the default mode for this repo.

@HuaizhengZhang could you resolve the .gitignore conflicts?

It is not super easy to do distributed training, but there are specialized groups, e.g. check https://github.com/learning-at-home/hivemind