OpenRouterTeam
generateObject functionality still broken with Claude Sonnet models in latest stable release
Issue Summary
The generateObject functionality continues to fail when using Claude Sonnet models with the OpenRouter AI SDK, despite previous attempts to fix this issue. This affects the latest stable versions and prevents structured object generation.
Demo Repository
I've created a minimal reproduction case demonstrating this issue: 🔗 https://github.com/rob-gordon/openrouter-generateobject-issue-demo
Environment
- @openrouter/ai-sdk-provider: ^1.1.2 (latest stable)
- ai: ^5.0.12 (latest stable)
- zod: ^4.0.17
- Model: anthropic/claude-3.5-sonnet
Expected Behavior
The generateObject method should successfully generate structured objects according to the provided Zod schema when using Claude Sonnet models.
Actual Behavior
The functionality fails (specific error details available by running the demo repo).
Related Issues
This appears to be related to issue #120 which was supposedly fixed by PR #132. However, the fix appears to be incomplete or the issue has regressed in the stable release.
The previous issue mentioned problems with JSON schema handling and response format specification, but these problems persist in the current stable version.
Reproduction Steps
- Clone the demo repository: https://github.com/rob-gordon/openrouter-generateobject-issue-demo
- Run bun install
- Set your OpenRouter API key
- Run bun run index.ts
- Observe the failure
I can confirm that generateObject() does not work with anthropic/claude-3.5-sonnet.
It works though with mistralai/mistral-small-24b-instruct-2501:free.
May or may not be a bug in the AI SDK, I reference this closed issue, just in case -> https://github.com/vercel/ai/issues/7358
After digging a bit deeper, it looks like none of the Anthropic models support structured outputs?
You can check for yourself with my model list tool, if you have Deno installed:
deno run -A jsr:@fry69/or-models --supports-structured-output --sort-by name
@fry69 I'm guess I'm not sure whether OpenRouter supports structured outputs for anthropic models, but I believe
that it's technically possible for them to support it because if you replace @openrouter/ai-sdk-provider with @ai-sdk/anthropic then generateObject works as expected.
// import { openrouter } from "@openrouter/ai-sdk-provider";
import { anthropic } from "@ai-sdk/anthropic";
import { generateObject } from "ai";
import { z } from "zod";
const PersonSchema = z.object({
name: z.string().describe("The name of the person"),
age: z.number().describe("The age of the person"),
occupation: z.string().describe("The occupation of the person"),
});
const result = await generateObject({
// model: openrouter("anthropic/claude-3.5-sonnet"),
model: anthropic("claude-3-5-sonnet-20240620"),
schema: PersonSchema,
prompt:
"Generate information about a fictional software engineer named Alex.",
});
console.log(result.object);
I guess I would expect using anthropic models via openrouter to be a drop-in replacement for using them directly, but maybe that's not the intended use case.
@fry69 I'm guess I'm not sure whether OpenRouter supports structured outputs for anthropic models
The supported parameters array in the OpenRouter API model list miss the response_format and/or structured_outputs flags for all Anthropic models. I guess that is why the OpenRouter API will treat Anthropic models as not capable of this and ignore the supplied schema/not forward it to the Anthropic API.
@louisgv Can you enlighten us what is going on here?
The mistral models are also no longer working after the upgrade to v5, while working fine in v4. Here is the ticket: https://github.com/OpenRouterTeam/ai-sdk-provider/issues/161
Mistral models (at those who offer the structured_output parameter) work for me.
Also, the linked test case does not seem to use the OpenRouter provider at all.
same for sonet generateObject just works maybe 1 out of 4 times. ps using anthropic provider
