[Question]: Inquiry About LLM Configuration for Accurate Knowledge Base Responses

[quang1603]

Self Checks

• [x] I have searched for existing issues search for existing issues, including closed ones.
• [x] I confirm that I am using English to submit this report (Language Policy).
• [x] Non-english title submitions will be closed directly ( 非英文标题的提交将会被直接关闭 ) (Language Policy).
• [x] Please do not modify this template :) and fill in all the required fields.

Describe your problem

Dear Team,

I am implementing a Q&A system using an LLM model combined with a knowledge base. However, I have noticed that the model sometimes modifies the content or adds extra information beyond what is available in the knowledge base, leading to inaccuracies in the responses.

I would like to ask about the key parameters that need to be adjusted to ensure that the responses strictly align with the knowledge base and do not introduce unintended modifications or additional content. Some specific issues I am encountering include:

The model tends to paraphrase rather than provide verbatim responses from the knowledge base. Some information that is not present in the source data appears in the responses. When asking the same question multiple times, the results vary even though the underlying data remains unchanged. I would greatly appreciate your guidance on how to optimize the model's configuration to improve response accuracy.

Best regards,

[KevinHuSh]

1. Switch to a larger LLM.
2. Refine the prompt.
3. Increase the threshold of similarity and decrease the Top N.
4. In a session, one question has been asked multiple times, then LLM will be confused its answer might be wrong.