Open-Assistant
Open-Assistant copied to clipboard
Add ability to tag tasks as `unanswerable`
It's very likely we'll create a task that can't be answered because it's not sensible or complete garbage.
We should add a button and API piping to ensure a user can mark any task (not individual messages but the task itself) as not answerable. This should ultimately prevent the task being served to other users (perhaps after some threshold of users mark it as unanswerable) and provide negative feedback to whomever created the messages in the task.
(re: end goal) Might also be worth considering number of users that skip the task as well as mark it unanswerable.
I like that as a signal and have used it before.
Depending on how the backend implements the support, that could be doable with this issue or separately. We currently don't report anything to the backend when we skip but I could imagine the current endpoint we're using take in alternative fields like skip
or tag_as_garbage
. Then a skip or unasnwerable button would operate the same as submitting an answer.
UI-wise how about something like this (I'm wary of too many buttons and clutter on mobile), the wording is just an example.
I like this UI.
I'll take this on, it'll give me a change to learn some of the backend code.
Issue #344 could be useful in handling this. That's a strategy for applying a label to an entire task. That label set could include skip
and unanswerable
.
During the front end teams call there was some discussion about ack
-ing and nack
-ing and it thought that maybe the nack
could include a reason for rejecting/skipping the task. In order to implement this task (flagging tasks that can't be completed and should not be shown to other users) we need to send feedback from the frontend to the backend and from what I found we don't yet have a suitable API call.
I have the following questions that will help me with this task:
- Should we be sending
nack
events when skipping? - Should we add a reason to the
nack
(no-reason, cant-complete, cant-read-language)? - If we continue to auto
ack
in the website, or choose to wait for the user to accept and thenack
, can we laternack
if the user skips after accepting (better that they can skip at any time for any reason so we at least get a signal and can pass the task to another user). - Is there a better API call to add this information to, or existing one I've missed?
EDIT: Just noticed that I might have been wrong and it looks like the backend does accept a reason for a nack
.
From what I understood in the meeting we can ack
and then nack
if needed.
So I think we can:
- Fetch a task and implicitly
ack
- If the user skips then submit a
nack
with a reason - If the user answers, then submit an the answer.
I had to disable the ack
ing temporarily as the backend currently rejects a nack
after an ack
, but this is what it looks like so far (the backend is receiving the event, but only one task in the frontend is setup); messages are still placeholders.
This is looking pretty smooth. Is it ready for review?
There are some fundamental questions about the how skipping should work in the discord website channel that we need to resolve. The code is at a proof-of-concept stage, it needs some cleanup on the routing parts and the only task page that uses it is user_reply, it needs to be added to the other task pages or made reusable.
I think what we have is sufficient for an MVP. There's a small fix I'll make in a tidying PR.