R2R icon indicating copy to clipboard operation
R2R copied to clipboard

Agent Hallucinates in RAG Mode Without Returning Search Results or Tool Call Traces

Open belynext opened this issue 5 months ago • 1 comments

Steps to Reproduce: 1. Set the task prompt with variables {date}, {max_tool_context_length}, and {document_context} (likely unrelated). 2. Set mode to rag. 3. Provide the agent access to the following tools: • search_file_descriptions • search_file_knowledge • get_file_content

Observed Behavior: • The agent generates responses that include information not present in any of the documents in the collection. • No tool call results or search result traces are returned by the server, making it impossible to trace how the agent arrived at its answer. • This breaks transparency and severely limits our ability to debug hallucinations or inspect retrieval steps.

Expected Behavior: • Every agent answer should include tool call results or search traces, especially in RAG mode, to allow SDK-side verification and handling of hallucinations.

Proposal: • Modify the server to return all tool call outputs, including the results from search or content retrieval tools. • If no tool was called or results were empty, that should also be clearly indicated in the response payload.

Impact: • Currently limits production use of RAG mode in agent pipelines. • Blocks us from building debugging or trust layers on top of agent behavior.

Thank you in advance

belynext avatar May 09 '25 08:05 belynext