Tobias Pitters

Add entities, which are refered in model/model_training/custom_datasets/formatting.py

Implement Dataset Entry for Datasets

20

We have a new `DatasetEntry` class which helps us to generalize over datasets and enforce a common formatting. We need to implement this class for a couple of more datasets:...

Implement RM Mode for DatasetEntry

1

Currently the `DatasetEntry` class cannot be used for reward model training, and a `NotImplementedError` is thrown, see https://github.com/LAION-AI/Open-Assistant/blob/main/model/model_training/custom_datasets/formatting.py#L88. This should be fixed as soon as possible. The response structure can...

#2827 In the [PR to introduce RM for the dataset entry class](https://github.com/LAION-AI/Open-Assistant/pull/2867) I forgot that if we have RM, we'll have multiple answers per question so `[Q1, (A1, A12)]` but...

Can't install model_training

6

Since the commit https://github.com/LAION-AI/Open-Assistant/pull/2206 I can't install model_training anymore. It looks like we try to install `ray` from a private aws artifactory via `trlx`. The error (while just trying to...

Update instruction datasets to return the `DatasetEntry` class. Also checked with `python check_dataset_appearances.py -d humaneval_mbpp_codegen_qa --cache_dir .cache --mode sft` but did not find anything.

Add system tag for each answer in a back and forth conversation. So we have to convert `[Q1, A1, Q2, A2]` to `q1attrib1a1q2attrib2a2` This also includes changing the prompter and...

WIP: add system tag for each question

3

closes #2972

This is the suggestion for a major refactoring of the data preprocessing in the model_training `trainer_sft`, `trainer_rm` and `trainer_rl` logic. The crucial point here is 4. but we need the...

closes #8683