holmesgpt
holmesgpt copied to clipboard
robusta-dev
View data from tools that is being sent to LLM
What would you like to be added?
An easy way to view exactly what data is being sent to the LLM. For cases when the data is being truncated, we want to see what data is being sent and what data was cut out.
Why is this needed?
For better troubleshooting of Holmes ask for scenarios requiring a lot of data (e.g. pod logs, DNS traces), and to see how we can improve the truncation logic to be more intelligent.
It might be overkill here, but eventually it would be nice to have a 2 pane view - with output on the left and tools on the right. We already have a dependency on Rich that can do it. Textual might be a good fit too.
Another option could be logging each tool call it to file (when a cli flag is passed) and showing in cli output where to find that file.
Does the truncation take place after all of the tools have been called? In that case, it would be valuable to compare the full results from each tool call vs. what data was ultimately passed to the LLM.
Good idea.
Regarding truncation, it's both. A typical session can look like this:
user: question llm: please call tools a,b,c user: here are results a, b, c llm: please call tools x, y, z user: here are the results for x, y, z
So we make a truncation decision for a, b, and c all at once. Then for x, y, and z all at once. (This assumes the LLM support parallel tool calling where we get multiple tool calls at once - most now do.)