Open-Assistant icon indicating copy to clipboard operation
Open-Assistant copied to clipboard

Determine list of labels

Open olliestanley opened this issue 2 years ago • 2 comments
trafficstars

At some point we need to finalise the list of valid text labels. Current list is here: https://github.com/LAION-AI/Open-Assistant/blob/main/oasst-shared/oasst_shared/schemas/protocol.py#L263

It has been mentioned that we may want to add "unhelpful" and "inappropriate".

I would advocate removing "ad_hominem" and "insult" (as they are essentially covered by "toxicity") as well as "based", "cringe", "formal", and maybe "slang" (as these are highly subjective). It's also unclear whether we need both "helpful" and "informative" as separate labels.

olliestanley avatar Jan 07 '23 15:01 olliestanley

The text_label enum needs more love. We should only have a low number of labels and their meaning must be clear. We also need display_text and help_text for all of them. Currently they are an Enum in code .. we should consider moving them into a yaml or json file.

andreaskoepf avatar Jan 07 '23 21:01 andreaskoepf

I have created #509 with some initial revisions to the enum, trimming the number of categories and providing clearer definitions for those remaining.

Would be good to hear views from anyone on what we might be missing or if anything else could be removed

Edit: One we might want to have could be "religious_content"?

Edit: #509 is now merged but leaving this issue open for if anyone has any further comments

olliestanley avatar Jan 07 '23 22:01 olliestanley