Open-Assistant
Open-Assistant copied to clipboard
Handle case when all prompts are factually inaccurate
Example
In this case it would be good to allow the labeler to indicate all the answers are incorrect, or to add their own, factually correct answer.
Best you currently can do is downgrade (thumbs down) and report the parent-message.
For this you can optionally skip the task, downvote both answers. But also from a data training perspective, ranking them is still valuable.
I'm going to change this to a documentation bug and request that someone fills out these details in the relevant docs page.
To clear this I think it's sufficient to create a markdown file in docs/docs/tasks/rank_assistant_replies.md
and add some instructions on what to do in corner cases. Then update website/src/components/Tasks/TaskTypes.tsx
and fill out the appropriate help_link
field.
This has been implemented