mudler
[Responses API] New OpenAI API and Tooling for an Agentic Year
Based upon OpenAI's recent Agentic Updates: https://openai.com/index/new-tools-for-building-agents/
Is your feature request related to a problem? Please describe. The local-ai project currently does not support the latest Responses API and built-in tools (web search, file search, computer use) introduced by OpenAI for building AI agents. This limits users who want to leverage these advanced capabilities with open source language models locally, preventing them from building more autonomous and efficient AI agents without relying on OpenAI's cloud-based services.
Describe the solution you'd like I would like the local-ai project to implement the Responses API, including support for the built-in tools such as web search, file search, and computer use (or integrations with other open source tools to satisfy those things). This would enable users to build and manage AI agents locally using open source models, mirroring the functionality provided by OpenAI's API. Ideally, local-ai would provide a compatible endpoint that allows these tools to operate with local resources or user-configured services, maintaining the project's focus on local execution.
Additional context OpenAI's new Responses API and built-in tools are designed to enhance AI agents' autonomy and functionality, allowing them to perform complex tasks like web searches, file handling, and computer interactions. Integrating these into local-ai would empower users to create advanced AI applications locally with open source models, preserving the benefits of privacy and control. Additionally, OpenAI plans to deprecate the Assistants API in favor of the Responses API by mid-2026, making this update crucial for maintaining compatibility with OpenAI's evolving standards. For more details, see the announcement: https://openai.com/index/new-tools-for-building-agents/.
Request generated thanks to a breakdown by Grok 3 :)
Thanks for opening this - yes I think it totally aligns with LocalAI's mission. Looking forward to bring more parity with OpenAI featureset.
This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 5 days.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
The OpenAI response API https://platform.openai.com/docs/api-reference/responses/create#responses-create-tools now support the code interpreter tool in addition to Web Search, File search, image generation, MCP tools.
It would be awesome if LocalAI could implement this API with all these tools. _ Web search could be down with tavily _ Code interpreter, generated python code could be executed in the jupyter/scipy-notebook:latest docker image for sandboxing.
While trying to keep LocalAI implementation lean, the responses API are actually supported by the LocalAGI sister project https://github.com/mudler/LocalAGI .
This was made on purpose to avoid saturating LocalAI with features that are more agentic and needs more opinionated way of working. I'm happy to revisit this - but on one side I think having LocalAGI separated is still a good separation of concerns.