Open-Assistant
Open-Assistant copied to clipboard
LAION-AI
Re-evaluate the design and descriptions of tasks for a better understanding
From comment https://github.com/LAION-AI/Open-Assistant/pull/127#pullrequestreview-1232736041
Consider adjusting the tasks' description and possibly design
For example, the rank_user_replies task currently shows different replies, but it does not show what is being replied to? critical context is missing.
Go through all tasks and evaluate whether a change is required and what:
- Assistant reply & User reply
- Rank Assistant replies & Rank User replies
- Rank initial prompts
- ~Summarize story & Rank summary~ (not part of MVP)
yes you're right, there should be the entire thread up to that point so the rater knows the context of the replies they're asked to rank!
Assistant reply & User reply
Currently the "speaker" is color coded, maybe it makes sense to add some text to show the speaker? Mockup (design could be better of course):
Maybe using tags and icons?
only color-coding is not really suitable, because it needs a legend and potentially people with visual impairments could have trouble discriminating.
"User" is called "Prompter" to avoid confusion internally, but towards the actual user, idk what makes more sense.
I like the approach with avatars the most, maybe even avatars on different sides like many chat-apps
I think avatars is the clearest and most representative of what the conversations actually look like, thus making it the clearest for a new user to understand.
For bonus points we could AI generate some avatars to represent the prompter (kindof a joke, but maybe a lower effort set of N pregenerated avatars wouldn't actually be too much work :smile: ).
We can use the "profile Picture" for the prompter. The prompter is already familiar with the Icon.
How are we on this issue? I feel like we haven't discussed this problem in a while so it's probably solved. @AbdBarho can you evaluate and close if appropriate?