vercel
[Feature Request]: Code snippet execution instead of individual tool-calling
Description
This pattern is known as CodeAct. Cloudflare recently published a blog post and added this pattern, calling it "Code Mode", not to be confused with ai-sdk's codemods.
The core idea is that tool-calling is unnatural to LLMs. There is a lot more training data for writing code, instead of tool-calls using special tokens. With V8 isolates, it is possible to evaluate LLM-generated code safely. So, we can generate a Typescript API from the tools MCP discovers, and ask LLM to write a code snippet instead of a single tool call. And then we execute this snippet safely in an isolate.
As the paper shows, there are many advantages of this approach. But Typescript and Isolates make it pragmatic.
It would be great to have this ability in ai-sdk natively, irrespective of how tools are discovered.
In essence, this feature request is about re-implementing codeact implementation from Cloudflare's Agent SDK in ai-sdk. AFAIK, Cloudflare's implementation only supports sandboxes in their infra. But there's no reason why code snippet execution cannot be done locally (in NodeJS). workerd runs locally but is an entire runtime and might be unnecessary.
AI SDK Version
ai: 5.0
Code of Conduct
- [x] I agree to follow this project's Code of Conduct
For people coming here, sandboxing is required for this to be production-ready and not be a hell of a security headache. I'm implementing this with Vercel Sandboxes rn. It's the most practical if you run everything in Vercel.
Anthropic recently published a blog post about this approach.
They also open-sourced their sandboxing runtime which Claude Code on the Web uses and published a post about it.
@carloslfu codeact + Vercel Sandboxes is something we want to look into once we ship v6 stable. If you have any findings / insights, please share!
New open-source and more portable tooling for code mode is now emerging:
- https://portofcontext.com/
- https://github.com/aberemia24/code-executor-MCP