experimental_output + tools in generateText breaks or continues forever with gpt-oss

[jeffmylife]

Description

My goal is to have tools be used and have a structured output. I'm using gpt-oss-120b via the AI Gateway.

The models that worked as expected were google/gemini-2.5-flash and openai/gpt-5-mini while the anthropic models wouldn't call the tools (but wouldn't error).

I attached a script in case you want to test this!

error case 1

This should work but does not:

const result = await generateText({
    model: llm,
    system: systemPrompt,
    prompt: prompt,
    experimental_output: Output.object({ schema: outputSchema }),
    stopWhen: stepCountIs(4),
    tools: tools
  });
// NoObjectGeneratedError [AI_NoObjectGeneratedError]: No object generated: response did not match schema.
// OR
// will occasionally complete but without calling tools

The detailed error:

NoObjectGeneratedError [AI_NoObjectGeneratedError]: No object generated: response did not match schema.
    at Object.parseOutput (/Users/jeffrey/Projects/outnurture/node_modules/.pnpm/[email protected][email protected]/node_modules/ai/src/generate-text/output.ts:117:15)
    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
    at async fn (/Users/jeffrey/Projects/outnurture/node_modules/.pnpm/[email protected][email protected]/node_modules/ai/src/generate-text/generate-text.ts:595:27)
    ... 2 lines matching cause stack trace ...
    at async main (/Users/jeffrey/Projects/outnurture/.ignore/aisdk-exp-output-bug.ts:53:18) {
  cause: _TypeValidationError [AI_TypeValidationError]: Type validation failed: Value: {"toolCall":{"name":"requiredTool","arguments":{"text":"A Shiba smiles with fox-like grace,\nCurled tail dancing, quick in pace.\nEyes that sparkle, sharp and sly,\nChasing breezes, under sky."}}}.
  Error message: [
    {
      "expected": "string",
      "code": "invalid_type",
      "path": [
        "summary"
      ],
      "message": "Invalid input: expected string, received undefined"
    }
  ]

error case 2

I found that if I add toolChoice: "required" it successfully calls the tool but doesn't generate the "stop" token and continues to attempt to call the tool with a NoSuchToolError until it reaches the stopWhen. In this condition it does succeed overall.

Here's the detailed error

tool calls [
  {
    type: 'tool-call',
    toolCallId: 'call_pngfxgdZpdend9W45aHCxVcy',
    toolName: '<|constrain|>json',
    input: {
      summary: 'A brief, vivid poem describes a Shiba Inu with a fox-like smile, a curled tail that dances, sparkling sly eyes, and a quick, breezy demeanor under the sky.'
    },
    dynamic: true,
    invalid: true,
    error: NoSuchToolError [AI_NoSuchToolError]: Model tried to call unavailable tool '<|constrain|>json'. Available tools: requiredTool.
        at doParseToolCall (/Users/jeffrey/Projects/outnurture/node_modules/.pnpm/[email protected][email protected]/node_modules/ai/src/generate-text/parse-tool-call.ts:104:11)
        at parseToolCall (/Users/jeffrey/Projects/outnurture/node_modules/.pnpm/[email protected][email protected]/node_modules/ai/src/generate-text/parse-tool-call.ts:34:20)
        at <anonymous> (/Users/jeffrey/Projects/outnurture/node_modules/.pnpm/[email protected][email protected]/node_modules/ai/src/generate-text/generate-text.ts:453:17)
        at Array.map (<anonymous>)
        at fn (/Users/jeffrey/Projects/outnurture/node_modules/.pnpm/[email protected][email protected]/node_modules/ai/src/generate-text/generate-text.ts:452:16)
        at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
        at async <anonymous> (/Users/jeffrey/Projects/outnurture/node_modules/.pnpm/[email protected][email protected]/node_modules/ai/src/telemetry/record-span.ts:18:22)
        at async generateText (/Users/jeffrey/Projects/outnurture/node_modules/.pnpm/[email protected][email protected]/node_modules/ai/src/generate-text/generate-text.ts:273:12)
        at async main (/Users/jeffrey/Projects/outnurture/.ignore/aisdk-exp-output-bug.ts:53:18) {
      cause: undefined,
      toolName: '<|constrain|>json',
      availableTools: [Array],
      Symbol(vercel.ai.error): true,
      Symbol(vercel.ai.error.AI_NoSuchToolError): true
    }
  }
]

---

Related issues

#8009

AI SDK Version

• ai: 5.0.27
• @ai-sdk/gateway 1.0.4

Script to test this: aisdk-exp-output-bug.ts

Outputs: runs.txt

Code of Conduct

• [x] I agree to follow this project's Code of Conduct

[gr2m]

Is the problem specific to your prompt? If you could share a minimal reproducible test case, it will help us prioritize the issue

[jeffmylife]

The test case is linked in the issue aisdk-exp-output-bug.ts

It could be due to that prompt. I did a few tests to try to help the structured output but didn't work only for oss.

[gr2m]

The test case is linked in the issue aisdk-exp-output-bug.ts

sorry I missed that 🤦🏼 I'll have a look

[gr2m]

I ran your code a few times but could not reproduce the problem yet.

What zod version are you using? We recommend using zod4

import { z } from "zod/v4";

If you can't do that, at least update to the latest zod and import v3 explicitly

import { z } from "zod/v3";

I've heard reports of similiar problems that most likely originated from the JSON Schema that we sent to the API. Could you please log out

console.dir({body: result.request.body}, { depth: null });

I'm particularly interested in responseFormat

   responseFormat: {
      type: 'json',
      schema: {
        type: 'object',
        properties: { summary: { type: 'string' } },
        required: [ 'summary' ],
        additionalProperties: false,
        '$schema': 'http://json-schema.org/draft-07/schema#'
      }
    },

[gr2m]

I see the error now, after setting toolChoice: "required" 👍🏼

[jeffmylife]

What zod version are you using? We recommend using zod4

Using zod 4.0.17

and then added "zod/v4" to no effect.

I'm particularly interested in responseFormat

Here's what I got from mine.

 responseFormat: {
    type: 'json',
    schema: {
      '$schema': 'http://json-schema.org/draft-07/schema#',
      type: 'object',
      properties: { summary: { type: 'string' } },
      required: [ 'summary' ],
      additionalProperties: false
    }
  }

Full results from the run:

output-w-request-body.txt

[songkeys]

Any solution to this issue?

[MartinCura]

fyi also bumped into this, and removing toolChoice solved it; i'm now just coercing the LLM with the prompt and it's fine

[songkeys]

removing toolChoice solved it;

but the op's script does not include toolChoice as well? Mine without toolChoice also triggers this error.

After upgrading to 5.0.68 for this commit: https://github.com/vercel/ai/commit/f733285b4a1bc1395a17a7f8bce171016f3f9ba1. The error changes to AI_NoOutputSpecifiedError

---

edit: I took another look at the commit. Looks like the behavior is expected? - If a tool is provided, the llm won't parse schema ouput. I.e. experimental_output can't be used together with tools.

[MartinCura]

but the op's script does not include toolChoice as well? Mine without toolChoice also triggers this error.

Then probably my issue was similar but not the same. Haven't tried latest, doesn't seem worth it to bother. Good luck with your issue.

[jeffmylife]

I had some time to look into it more. This might not be an issue with aisdk. It might be upstream at the gateway level and in particular with handling GPT-OSS's harmony protocol (just a guess).

I added some debug logging and found

content: [
    { type: 'reasoning', text: '...' },
    { type: 'text', text: '{"fiveWordSummary":"..."}' },  // ← TEXT, not tool-call
    { type: 'text', text: '{"summary":"..."}' } 
  ]

So it seems that while the model is correctly calling a tool, the format of the tool isn't in the OpenAI spec like this:

 content: [
    { type: 'tool-call', toolName: 'requiredTool', toolCallId: 'call_123',
      args: {"fiveWordSummary":"..."} }
  ]

[makispl]

@jeffmylife have you found any solution to this?

[gerardatkonvo]

We are experimenting high failure rates with gpt-oss-120b-1:0 in generateObject calls. AI_NoObjectGeneratedError

[makispl]

I've found a way to avoid those issues with:

"error": "No output specified.",
"errorType": "NoOutputSpecifiedError",

as follows, by disabling tools so the model must answer after couple steps:

        const { experimental_output: sampleOutput } = await generateText({
          model,
          prompt: `prompt....`,
          tools: { sampleTool: createSampleTool(format) },
          stopWhen: stepCountIs(15),
          prepareStep: async ({ stepNumber }) => {
            // After a couple of steps, disable tools so the model must answer
            if (stepNumber >= 3) {
              return {
                activeTools: [] as const
              };
            }
            return {};
          },
          onStepFinish: ({ finishReason }) => {
            logger.info(`finishReason = "${finishReason}"`);
          },
          experimental_output: Output.object({ schema: sampleOutput Schema })
        });

[jeffmylife]

Ok things have changed since my original post.

When using gpt-oss-120b with default gateway params I get:

--------------------------------
starting generateText
--------------------------------
stepped with finishReason tool-calls
tool calls []
tool results []
--------------------------------
result.text {"text":"A Shiba smiles with fox-like grace,\nCurled tail dancing, quick in pace.\nEyes that sparkle, sharp and sly,\nChasing breezes, under sky."}{"summary":"The poem depicts a lively Shiba dog, describing its fox-like smile, energetic tail, sparkling sharp eyes, and its playful, breezy nature."}

Notice that we have two concatenated JSONs in the resulting text. If you attempt to extract the result.experimental_output you get the familiar AI_NoOutputSpecifiedError error:

NoOutputSpecifiedError [AI_NoOutputSpecifiedError]: No output specified.
    at DefaultGenerateTextResult.get experimental_output (/Users/jeffrey/Projects/outnurture/node_modules/.pnpm/[email protected][email protected]/node_modules/ai/src/generate-text/generate-text.ts:843:13)
    at main (/Users/jeffrey/Projects/outnurture/.ignore/aisdk-exp-output-bug-og.ts:83:67)
    at process.processTicksAndRejections (node:internal/process/task_queues:103:5) {
  cause: undefined,
  Symbol(vercel.ai.error): true,
  Symbol(vercel.ai.error.AI_NoOutputSpecifiedError): true

the culprit(s)

Upon further inspection into providerMetadata, I found that some providers (cerebras & groq) 400 error when using tools and JSON mode. When failed this way, the gateway falls back to other providers. Unfortunately, some provider (parasail, bedrock, baseten) requests succeed but are consistently broken at agent runtime. There's only one provider that succeeds at generating the object: fireworks.

Here's what the provider metadata looks like:

providerMetadata {
  "gateway": {
    "routing": {
      "originalModelId": "openai/gpt-oss-120b",
      ...
      "attempts": [
        {
          "provider": "cerebras",
          "internalModelId": "cerebras:gpt-oss-120b",
          "success": false,
          "error": "\"tools\" is incompatible with \"response_format\"", // not allowed
          "statusCode": 400
        },
        {
          "provider": "groq",
          "internalModelId": "groq:openai/gpt-oss-120b",
          "success": false,
          "error": "json mode cannot be combined with tool/function calling", // not allowed
          "statusCode": 400
        },
        {
          "provider": "baseten",
          "internalModelId": "baseten:openai/gpt-oss-120b",
          "success": true, // success but will fail to generate parseable object.
          "statusCode": 200
        }
      ],
      ...
}

What this tells me is that the ability to do JSON mode + tool calling with gpt-oss-120b is determined at the provider-level. I looked into the attempted results generated by each provider and found that they are each unique in their failures. For example, the bedrock implementation generates the concatenated JSONs I mentioned at before.

TL;DR

✅ If you're using gpt-oss-120b and want to have experimental_output with tool calling then set your provider options to only use fireworks:

  const providerOptions = {
    gateway: {
      order: [
        "cerebras", // 400 error --> fallback
        "groq", // 400 error --> fallback
        // "parasail", // 200 --> unsuccessfully generates object
        // "bedrock", // 200 --> unsuccessfully generates object
        // "baseten", // 200 --> unsuccessfully generates object
        "fireworks", // 200 --> successfully generates object 🎉
      ],
    },
  };

---

Attached is a script which can be used for reproducing my findings.

aisdk-exp-output-bug.ts