openai-node
openai-node copied to clipboard
feat: Add `afterCompletion` callback option for `runTools` to enable easily building multi-model / multi-agent flows
- [x] I understand that this repository is auto-generated and my pull request may not be merged
Changes being requested
I'd like to propose adding an afterCompletion hook as an option to runTools. This enables building powerful multi-model and multi-agent functionality easily by providing a simple entrypoint at which you can analyze runner.messages and optionally carry out a separate LLM flow and then inject one or more messages (or modify messages) in order to, for example, inject some targeted web research to help your agent overcome a problem it's encountering.
I've updated tests, added an example, and ensured everything is correctly typed.
My loom linked below shows how I'm using this to great effect in Otto Engineer, an autonomous coding system.
Additional context & links
Note that I tried to get this to work via tool definitions and prompting but got very poor results. I could almost never get the model to perform web research, and when I did, the prompt and tools had to be shaped so aggressively around this goal that it degraded performance of everything else. So I started thinking of ways to decompose this into separate problems and entirely remove web research from the prompt/tools so the model would not have to think about it and could focus purely on the already complex task of implementing and testing code changes.
I came up with an idea to periodically inspect the chat thread during a runTools call, and I initially attempted to do it via tricky Promise resolution combined with runner.abort + starting a new runTools call and modifying messages, but it was painfully complex and near impossible to correctly update the chat messages without accidentally injecting my web research message in the wrong place (prior to tool call outputs). (Controlling promises via the emitted events is hazardous as, of course, they can happen out of sequence, and it's hard to make sure you've captured all in progress tool calls / messages before injecting a message.)
Here's a demo loom where I show how I use this in my app.
Thank you
I absolutely love having static types around your API and all the goodness this library gives me! I get a TON of mileage out of runTools and want to contribute and help this library thrive! Thank you 🙏🏻