brainlid
Proposal: Separate the Step Between Tool Result and Assistant Message
I’d like to propose separating the step between the tool result and the assistant message in this library.
Currently, even when running in the default mode, after execute_tool_calls/1 is called, the next step do_run/1 is executed immediately, making it impossible for me to intervene or customize anything in between.
My use case is that when the tool_result returns a result for routing, I want to directly use this data (specifically the processed_content) to render a custom UI for the user at that point. This approach would allow me to provide a much faster and richer user experience by returning a tailored UI immediately after receiving the tool_result.
In my view, the entire flow—message -> tool_call -> execute (tool result) -> message—should be treated as a sequence of equal steps. If we make these steps equally accessible, users of this library will gain much more control over each stage, enabling a variety of advanced use cases.
Looking forward, I also hope to add support for the while_needs_response mode, where a function could be provided to determine at each step whether to proceed or pause, giving library users even more flexibility.
I believe that to achieve this, the library needs to treat each step in the chain as an equal unit, as described above. If the proposal is accepted, I am willing to work on this myself and contribute the changes to the library.
Thank you for considering this enhancement!
I think https://github.com/brainlid/langchain/commit/459a392cb3bceb73b6d2f6980c885b52c9ec301c might address the issue I was facing. I’ll test it out and report back.
After checking the changes, I realized that the new behavior always retries until the specific tool is used. It doesn’t work for my use case, as I also need to support generic tool usage.
Hi @nallwhy!
I have a couple thoughts.
If you use the run without a mode, it stops at each received message. At this point you could make any adjustments or determinations then optionally continue the chain.
As for your rich UI idea, I do this by including a process to notify in the context that a function/tool receives. It could be a LiveView, PubSub, whatever. But it allows me to async reflect what's going on.
Thanks for the reply!
As you suggested, I’m currently using run without a mode and handling the control flow myself. It did take a bit of work to implement the logic for when to continue and when to intercept, but it’s working well for my use case.