Message drafts

[someone13574]

closes #2931 (slightly changed goal based on advice from the discord, generate full messages, not 'x' tokens. Full messages are more useful data)

• [x] Create draft selection UI
• [x] Draft inference
• [x] Option to regenerate drafts and serve 3 new ones
• [x] Remember last viewed sibling message
• [x] Store selected draft training data for RLHF
• ~~[ ] Disable drafts when queue is too long / server is under load~~ (Suggested to leave to next PR in the discord)
• [x] Draft markdown rendering
• [x] 'Used plugin' UI for drafts
• [x] Resolve merge conflicts

[github-actions[bot]]

:x: pre-commit failed. Please run pre-commit run --all-files locally and commit the changes. Find more information in the repository's CONTRIBUTING.md

[github-actions[bot]]

:x: pre-commit failed. Please run pre-commit run --all-files locally and commit the changes. Find more information in the repository's CONTRIBUTING.md

[github-actions[bot]]

:x: pre-commit failed. Please run pre-commit run --all-files locally and commit the changes. Find more information in the repository's CONTRIBUTING.md

[github-actions[bot]]

:x: pre-commit failed. Please run pre-commit run --all-files locally and commit the changes. Find more information in the repository's CONTRIBUTING.md

[github-actions[bot]]

:x: pre-commit failed. Please run pre-commit run --all-files locally and commit the changes. Find more information in the repository's CONTRIBUTING.md

[olliestanley]

Thanks, I think this mostly makes sense from inference backend perspective, I left a couple of questions and one change request

[github-actions[bot]]

:x: pre-commit failed. Please run pre-commit run --all-files locally and commit the changes. Find more information in the repository's CONTRIBUTING.md

[notmd]

Also, can we make an option to allow users to choose whether they want to generate drafts or not?

[github-actions[bot]]

:x: pre-commit failed. Please run pre-commit run --all-files locally and commit the changes. Find more information in the repository's CONTRIBUTING.md

[someone13574]

Thanks a lot for this important feature. I would appreciate feedback to the following points:

1. It seems as if the draft generation would always be active when `NUM_GENERATED_DRAFTS > 1`. This will completely break any regular chat-experience and force the user to go through the message-selection process, even for simple "Hello" prompts. In general we definitely want to collect this form of feedback data but the question is if we really should make regular use of assistant impossible. If message alternatives would be generated when user presses regenerate or down-vote it would IMO be a much more unobtrusive experience and users would get additional option when they actually need them.

2. It is not clear to me how draft messages are tracked now in the database. Could you please explain the meaning the `active_message_id` column in `chat` table (and why is it a json field)?  How are drafts messages stored and referenced in the database?

1. I'll make the drafts less intrusive by only generating drafts if either the prompt is long (thus indicating a more complex request) or if the user regenerates a message (indicating that additional options would be beneficial). This should make the process less intrusive and act as a filter against data we really don't need comparisons for (trivial tasks or general conversation).
2. Draft messages are stored the same way as normal message and the comparisons are stored in an new table called message_eval where it has the id of the selected message and the id's of the messages it was selected over. The active_message_id stores the id of the last message in the currently visible thread so that the conversation can be loaded in the same state as it was when a user closes it. This is because all the unselected drafts are also added to the conversation as other threads and the last message for a given parent message wont always be the one the user was actually using, so loading onto that unused message instead of where the user actually was writing was very bad for ux, so I made it remember which thread was last selected. I'd be open to a better name. Also using a json was a mistake, I'll fix it.

[github-actions[bot]]

:x: pre-commit failed. Please run pre-commit run --all-files locally and commit the changes. Find more information in the repository's CONTRIBUTING.md

[github-actions[bot]]

:x: pre-commit failed. Please run pre-commit run --all-files locally and commit the changes. Find more information in the repository's CONTRIBUTING.md

[github-actions[bot]]

:x: pre-commit failed. Please run pre-commit run --all-files locally and commit the changes. Find more information in the repository's CONTRIBUTING.md

[github-actions[bot]]

:x: pre-commit failed. Please run pre-commit run --all-files locally and commit the changes. Find more information in the repository's CONTRIBUTING.md