Open-Assistant icon indicating copy to clipboard operation
Open-Assistant copied to clipboard

Add ability to tag tasks as `unanswerable`

Open fozziethebeat opened this issue 2 years ago • 11 comments

It's very likely we'll create a task that can't be answered because it's not sensible or complete garbage.

We should add a button and API piping to ensure a user can mark any task (not individual messages but the task itself) as not answerable. This should ultimately prevent the task being served to other users (perhaps after some threshold of users mark it as unanswerable) and provide negative feedback to whomever created the messages in the task.

fozziethebeat avatar Jan 03 '23 06:01 fozziethebeat

(re: end goal) Might also be worth considering number of users that skip the task as well as mark it unanswerable.

othrayte avatar Jan 05 '23 10:01 othrayte

I like that as a signal and have used it before.

Depending on how the backend implements the support, that could be doable with this issue or separately. We currently don't report anything to the backend when we skip but I could imagine the current endpoint we're using take in alternative fields like skip or tag_as_garbage. Then a skip or unasnwerable button would operate the same as submitting an answer.

fozziethebeat avatar Jan 05 '23 10:01 fozziethebeat

UI-wise how about something like this (I'm wary of too many buttons and clutter on mobile), the wording is just an example. image

othrayte avatar Jan 05 '23 11:01 othrayte

I like this UI.

fozziethebeat avatar Jan 05 '23 11:01 fozziethebeat

I'll take this on, it'll give me a change to learn some of the backend code.

othrayte avatar Jan 05 '23 11:01 othrayte

Issue #344 could be useful in handling this. That's a strategy for applying a label to an entire task. That label set could include skip and unanswerable.

fozziethebeat avatar Jan 05 '23 12:01 fozziethebeat

During the front end teams call there was some discussion about ack-ing and nack-ing and it thought that maybe the nack could include a reason for rejecting/skipping the task. In order to implement this task (flagging tasks that can't be completed and should not be shown to other users) we need to send feedback from the frontend to the backend and from what I found we don't yet have a suitable API call.

I have the following questions that will help me with this task:

  • Should we be sending nack events when skipping?
  • Should we add a reason to the nack (no-reason, cant-complete, cant-read-language)?
  • If we continue to auto ack in the website, or choose to wait for the user to accept and then ack, can we later nack if the user skips after accepting (better that they can skip at any time for any reason so we at least get a signal and can pass the task to another user).
  • Is there a better API call to add this information to, or existing one I've missed?

EDIT: Just noticed that I might have been wrong and it looks like the backend does accept a reason for a nack.

othrayte avatar Jan 06 '23 10:01 othrayte

From what I understood in the meeting we can ack and then nack if needed.

So I think we can:

  1. Fetch a task and implicitly ack
  2. If the user skips then submit a nack with a reason
  3. If the user answers, then submit an the answer.

fozziethebeat avatar Jan 06 '23 11:01 fozziethebeat

I had to disable the acking temporarily as the backend currently rejects a nack after an ack, but this is what it looks like so far (the backend is receiving the event, but only one task in the frontend is setup); messages are still placeholders. skip

othrayte avatar Jan 06 '23 14:01 othrayte

This is looking pretty smooth. Is it ready for review?

fozziethebeat avatar Jan 06 '23 22:01 fozziethebeat

There are some fundamental questions about the how skipping should work in the discord website channel that we need to resolve. The code is at a proof-of-concept stage, it needs some cleanup on the routing parts and the only task page that uses it is user_reply, it needs to be added to the other task pages or made reusable.

othrayte avatar Jan 07 '23 05:01 othrayte

I think what we have is sufficient for an MVP. There's a small fix I'll make in a tidying PR.

fozziethebeat avatar Jan 10 '23 03:01 fozziethebeat