Open-Assistant
Open-Assistant copied to clipboard
In "Label Assistant Reply" subdivide "Quality" into "Accuracy" and f.e. "Grammar"
I think it would be a good idea to subdivide the "Quality"-category in "Label Assistant Reply" into "Accuracy" and f.e. "Grammar" if that is possible because the term "Quality" is a relatively vague term. And I also think it would generally be a good idea to focus the training of Open Assistant more on accuracy as inaccurate outputs are one of the biggest issues of competitors like ChatGPT, etc.
Ok, I must admit though it's not always easy for people to asses the accuracy of an output and it's admittedly also possible that people might bring in their own biases meaning I would still keep answering the new categories "Accuracy" and "Grammar" optional and would perhaps suggest to the main-developers of Open Assistant to see the user-contributed "Accuracy"-assesment as merely a first step in checking the accuracy of assistant replies.
Ok, I was thinking about closing that issue again because I just found out that the dashboard also includes a function sometimes popping up called something like "Rank assistant reply" where you are apparently supposed to rank assistant replies based on accuracy however as the issue got two likes I will keep it open for now.
I don't think it's practical to change the labelling options with so much data already submitted, but maybe others have other views so we can leave this open for any further discussion
Going to close this as covered by #1412