Andreas Köpf

and https://www.together.xyz/

> I would take inspiration from earlier LAION success and lean more towards distributed pre-processing of training data. There is plenty of cost in things like the reinforcement learning rollouts...

The OA dataset has not been released. If you want to prepare training code you can look at a sample of 100 English trees here: https://github.com/Open-Assistant/oasst-model-eval/blob/main/model_eval/manual/data/en_100_tree.jsonl If you are interested...

Currently the website asks the backend via endpoints `api/v1/users/{id}/stats/{timeframe}` or `api/v1/leaderboards/{timeframe}` ... and uses the `UserScore`/`LeaderboardStats` model class defined in protocol.py The computation of the XP-level could for example be...

I am closing this to reduce confusion since we are effectively following a very different - much simpler plan.

I like Jordi's proposal. @CloseChoice do you think you could add this? .. similar to [summarization.py#L151](https://github.com/LAION-AI/Open-Assistant/blob/a570b04b93d1aa591369e41081c571046bbd7d3c/model/model_training/custom_datasets/summarization.py#L151) (just simpler for the different whitespace ..)

I close this for now since reverse-augmentation was causing more harm than it helped.

@MrlolDev could you please try to make pre-commit happy?

Any model that is based on HF-Transformers CaualLM should be usable without problems. If you have compute you could try fine-tuning a GPT JT model on the OA dataset.

> Would GPT JT be considered as an option as a pretrained model to fine-tune into the final model? Yes, that's definitely something we should try. Also probably the larger...