infiniflow
[Question]:
Self Checks
- [x] I have searched for existing issues search for existing issues, including closed ones.
- [x] I confirm that I am using English to submit this report (Language Policy).
- [x] Non-english title submitions will be closed directly ( 非英文标题的提交将会被直接关闭 ) (Language Policy).
- [x] Please do not modify this template :) and fill in all the required fields.
Describe your problem
Can developers collect users feedback by the thumbs button upon the output? If not, could you add this feature in next versions?
@konn-submarine-bu Sorry, this feature is not yet implemented and we don't have a timeline for its release.
Hi team, I saw in previous responses that the thumbs‑up/thumbs‑down feedback isn’t implemented yet. Do you have any plans or timeline to add this functionality, such as an API or way to access the data? Collecting user feedback would be very helpful. Thanks!
@chensid @konn-submarine-bu Hey guys, I could really use your help on this. After discussing with Kevin, we believe this feature could be valuable, like being used as a tool to evaluate RAG, but we’d love to hear why you need it and how you plan to use them. Your insights will help us make the right design decisions.
@chensid @konn-submarine-bu Hey guys, I could really use your help on this. After discussing with Kevin, we believe this feature could be valuable, like being used as a tool to evaluate RAG, but we’d love to hear why you need it and how you plan to use them. Your insights will help us make the right design decisions.
Thanks for the follow-up and for asking about my expectations.
As a developer using RAGFlow, a complete feedback feature around the thumbs button would ideally include three parts:
-
Data persistence for 👍/👎 feedback
Feedback should be stored so it can be referenced and analyzed later. -
API support for writing and querying feedback
This is essential for integrating with our product and for building dashboards or analytics. -
(Optional but preferred) Built-in UI support in RAGFlow
Having a native feedback button in the RAGFlow UI would make the experience consistent and easier for end users, while still allowing custom UIs to call the same API.
My goal is to use these feedback signals to evaluate answer quality and continuously optimize our system parameters (for example, retrieval and generation settings) and the knowledge base itself.
If the team is open to it, providing both the UI and the API would create a clean and complete feedback loop for us.
@chensid Thank you so much for such a detailed breakdown. This is incredibly helpful. How are you and your team currently doing evaluation? Are you using something like RAGAS or another framework?
Also, since you mentioned evaluating answer quality, I’m really curious: How do you and your team define a “high-quality” answer? What criteria matter the most to you?
tbh, Kevin and I really want to understand everyone’s thinking here, and we’d love to see whether there’s anything we can build to help.
@chensid Thank you so much for such a detailed breakdown. This is incredibly helpful. How are you and your team currently doing evaluation? Are you using something like RAGAS or another framework?
Also, since you mentioned evaluating answer quality, I’m really curious: How do you and your team define a “high-quality” answer? What criteria matter the most to you?
tbh, Kevin and I really want to understand everyone’s thinking here, and we’d love to see whether there’s anything we can build to help.
Thanks for the questions!
Internally we do use tools like RAGAS and DeepEval to cover the objective stuff — accuracy, relevance, completeness. But we also care a lot about user feedback (upvotes/downvotes), because that’s the part that actually tells us whether an answer feels useful or clear.
For us, a good answer is basically: accurate, on-topic, clear, complete enough, and genuinely helpful.
If you’re exploring anything that connects the metric side with real user signals, we’d love to check it out.