Support for OpenAI's new Responses API spec

[PierreMesure]

🚀 Feature

• Ability to expose assistants through an API that follows the Responses API standard and specs.

import litserve as ls

class SimpleLitAPI(ls.LitAPI):
    def setup(self, device):
        self.model = None

    def predict(self, prompt):
        # `prompt` is a list of dictionary containing role and content
        # example: [{'role': 'user', 'content': 'How can I help you today?'}]
        yield "This is a sample generated output"

if __name__ == "__main__":
    # Enable the OpenAISpec in LitServer
    api = SimpleLitAPI()
    server = ls.LitServer(api, spec=ls.ResponsesOpenAISpec())
    server.run(port=8000)

Motivation

OpenAI announced last week a new Responses API that contains the feature set of the Chat Completions API and replaces the Assistants API (the latter is now deprecated and will be sunseted mid-2026).

Azure already announced that they provide this API day one and considering OpenAI's position on the market, we can expect that it will become an industry standard as is the Chat Completions API today.

Additional context

• Official documentation for the Responses API
• Thread on OpenAI's developer forum
• Comparison between the Chat Completions API and Responses API
• A blog post from Simon Willison on the new API
• A thread on X by one of the creators of the API talking about its making

[aniketmaurya]

@deependujha @bhimrazy wanna take a look?

[PierreMesure]

In case you didn't know, OpenAI has an OpenAPI repository with all their API specs.

[bhimrazy]

@deependujha @bhimrazy wanna take a look?

Sure. @deependujha, Let us know incase you would like to take on this. Also, It seems like a significant feature, so we could break it down into multiple PRs, starting with text input.

[deependujha]

yeah sure. I'd like to take on this. Sorry for the delay.

[bhimrazy]

yeah sure. I'd like to take on this. Sorry for the delay.

No worries! Feel free to reach out if you have any questions along the way. 🚀

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.