Feature Request: Feedback for answers

[sidravi1]

Can we ask the user for feedback on if the response was useful? I'm imagining a 👍🏽 or 👎🏽 or similar buttons with the results. This will allow us to understand how well the solution is working out.

[yuhongsun96]

Definitely a great addition, a couple questions:

• Is this for metrics tracking so that the Danswer admin of a team can know how often an answer is good/bad?
• Would you like for this feedback to impact future document retrieval rankings (upweighting/downweighting) similar to endorsing/deprecating information sources?
• Would you like for this to impact model weights? This one is more complicated as either we need to support online learning (live model updates) or have a way to securely share data back with the user's approval so that new models can be trained offline.

[sidravi1]

Is this for metrics tracking so that the Danswer admin of a team can know how often an answer is good/bad?

Yes - this would be the immediate use of it. As an admin. I'd love to know how often the results are good / useful. Also knowing the kind of requests that it is doing poorly for will help us (a) share searching tips with users; (b) create better content, identify gaps in our knowledge base; (c) improve model performance - as discussed below.

Would you like for this feedback to impact future document retrieval rankings (upweighting/downweighting) similar to endorsing/deprecating information sources? Would you like for this to impact model weights? This one is more complicated as either we need to support online learning (live model updates) or have a way to securely share data back with the user's approval so that new models can be trained offline.

The second use case would be to learn from errors and improve performance. I guess a simple way would be to re-weight scores to form priors - some documents are more popular that other? I'm sure there are other sophisticated methods.