@effect/ai Ordering bug in `Prompt.fromResponseParts` causes Anthropic to return 400 Status

Open skoshx opened this issue 1 month ago • 0 comments

What version of Effect is running?

"@effect/ai": "^0.31.1", "@effect/ai-anthropic": "^0.21.1", "@effect/cli": "^0.71.0", "@effect/experimental": "^0.56.0", "@effect/platform": "^0.92.1", "@effect/platform-bun": "^0.81.1",

What steps can reproduce the bug?

Basically, the way that Prompt.fromResponseParts does is it returns assistant messages, followed by tool messages (tool results).

The problem is that Prompt.fromResponseParts includes tool calls in the assistant messages, which is fine, but the reduction logic somehow places a text part following a tool-call, which causes a 400 error from Anthropic API, because tool calls have to be IMMEDIATELY followed by either another tool call, or a tool result.

Code Snippet for a Repro:

import * as Prompt from "@effect/ai/Prompt";

const PARTS = [
  {
    type: "response-metadata",
    id: {
      _id: "Option",
      _tag: "Some",
      value: "msg_...",
    },
    modelId: {
      _id: "Option",
      _tag: "Some",
      value: "claude-3-7-sonnet-20250219",
    },
    timestamp: {
      _id: "Option",
      _tag: "Some",
      value: "2025-11-01T09:59:09.623Z",
    },
    metadata: {},
    "~effect/ai/Content/Part": "~effect/ai/Content/Part",
  },
  {
    type: "text-start",
    id: "0",
    metadata: {},
    "~effect/ai/Content/Part": "~effect/ai/Content/Part",
  },
  {
    type: "text-delta",
    id: "0",
    delta: "I",
    metadata: {},
    "~effect/ai/Content/Part": "~effect/ai/Content/Part",
  },
  {
    type: "text-delta",
    id: "0",
    delta: "'ll",
    metadata: {},
    "~effect/ai/Content/Part": "~effect/ai/Content/Part",
  },
  {
    type: "text-delta",
    id: "0",
    delta: " help you fix lint",
    metadata: {},
    "~effect/ai/Content/Part": "~effect/ai/Content/Part",
  },
  {
    type: "text-delta",
    id: "0",
    delta: " errors in your project",
    metadata: {},
    "~effect/ai/Content/Part": "~effect/ai/Content/Part",
  },
  {
    type: "text-delta",
    id: "0",
    delta: ". First, let's",
    metadata: {},
    "~effect/ai/Content/Part": "~effect/ai/Content/Part",
  },
  {
    type: "text-delta",
    id: "0",
    delta: " understand what kin",
    metadata: {},
    "~effect/ai/Content/Part": "~effect/ai/Content/Part",
  },
  {
    type: "text-delta",
    id: "0",
    delta: "d of linting errors exist",
    metadata: {},
    "~effect/ai/Content/Part": "~effect/ai/Content/Part",
  },
  {
    type: "text-delta",
    id: "0",
    delta: " and which",
    metadata: {},
    "~effect/ai/Content/Part": "~effect/ai/Content/Part",
  },
  {
    type: "text-delta",
    id: "0",
    delta: " files",
    metadata: {},
    "~effect/ai/Content/Part": "~effect/ai/Content/Part",
  },
  {
    type: "text-delta",
    id: "0",
    delta: " are affected.\n\nLet",
    metadata: {},
    "~effect/ai/Content/Part": "~effect/ai/Content/Part",
  },
  {
    type: "text-delta",
    id: "0",
    delta: "'s",
    metadata: {},
    "~effect/ai/Content/Part": "~effect/ai/Content/Part",
  },
  {
    type: "text-delta",
    id: "0",
    delta: " start",
    metadata: {},
    "~effect/ai/Content/Part": "~effect/ai/Content/Part",
  },
  {
    type: "text-delta",
    id: "0",
    delta: " by checking if",
    metadata: {},
    "~effect/ai/Content/Part": "~effect/ai/Content/Part",
  },
  {
    type: "text-delta",
    id: "0",
    delta: " there's",
    metadata: {},
    "~effect/ai/Content/Part": "~effect/ai/Content/Part",
  },
  {
    type: "text-delta",
    id: "0",
    delta: " a l",
    metadata: {},
    "~effect/ai/Content/Part": "~effect/ai/Content/Part",
  },
  {
    type: "text-delta",
    id: "0",
    delta: "inting comman",
    metadata: {},
    "~effect/ai/Content/Part": "~effect/ai/Content/Part",
  },
  {
    type: "text-delta",
    id: "0",
    delta: "d in your",
    metadata: {},
    "~effect/ai/Content/Part": "~effect/ai/Content/Part",
  },
  {
    type: "text-delta",
    id: "0",
    delta: " project,",
    metadata: {},
    "~effect/ai/Content/Part": "~effect/ai/Content/Part",
  },
  {
    type: "text-delta",
    id: "0",
    delta: " typically",
    metadata: {},
    "~effect/ai/Content/Part": "~effect/ai/Content/Part",
  },
  {
    type: "text-delta",
    id: "0",
    delta: " define",
    metadata: {},
    "~effect/ai/Content/Part": "~effect/ai/Content/Part",
  },
  {
    type: "text-delta",
    id: "0",
    delta: "d in package",
    metadata: {},
    "~effect/ai/Content/Part": "~effect/ai/Content/Part",
  },
  {
    type: "text-delta",
    id: "0",
    delta: ".json for",
    metadata: {},
    "~effect/ai/Content/Part": "~effect/ai/Content/Part",
  },
  {
    type: "text-delta",
    id: "0",
    delta: " JavaScript",
    metadata: {},
    "~effect/ai/Content/Part": "~effect/ai/Content/Part",
  },
  {
    type: "text-delta",
    id: "0",
    delta: "/",
    metadata: {},
    "~effect/ai/Content/Part": "~effect/ai/Content/Part",
  },
  {
    type: "text-delta",
    id: "0",
    delta: "Type",
    metadata: {},
    "~effect/ai/Content/Part": "~effect/ai/Content/Part",
  },
  {
    type: "text-delta",
    id: "0",
    delta: "Script projects:",
    metadata: {},
    "~effect/ai/Content/Part": "~effect/ai/Content/Part",
  },
  {
    type: "text-end",
    id: "0",
    metadata: {},
    "~effect/ai/Content/Part": "~effect/ai/Content/Part",
  },
  {
    type: "tool-call",
    id: "toolu_01QUntciPNGSht5XqytF3icG",
    name: "Tool1",
    params: {
      hello: "world",
    },
    providerExecuted: false,
    metadata: {},
    "~effect/ai/Content/Part": "~effect/ai/Content/Part",
  },
  {
    id: "toolu_01QUntciPNGSht5XqytF3icG",
    type: "tool-result",
    isFailure: false,
    "~effect/ai/Content/Part": "~effect/ai/Content/Part",
    name: "Tool1",
    result: {
      success: true
    },
    encodedResult: {
      success: true,
    },
    providerExecuted: false,
    metadata: {},
  },
  {
    type: "response-metadata",
    id: {
      _id: "Option",
      _tag: "Some",
      value: "msg_01GGUb7Zwpr9tuXiS6v2poo4",
    },
    modelId: {
      _id: "Option",
      _tag: "Some",
      value: "claude-3-7-sonnet-20250219",
    },
    timestamp: {
      _id: "Option",
      _tag: "Some",
      value: "2025-11-01T09:59:18.632Z",
    },
    metadata: {},
    "~effect/ai/Content/Part": "~effect/ai/Content/Part",
  },
  {
    type: "text-start",
    id: "0",
    metadata: {},
    "~effect/ai/Content/Part": "~effect/ai/Content/Part",
  },
  {
    type: "text-delta",
    id: "0",
    delta: "Let",
    metadata: {},
    "~effect/ai/Content/Part": "~effect/ai/Content/Part",
  },
  {
    type: "text-delta",
    id: "0",
    delta: "'s",
    metadata: {},
    "~effect/ai/Content/Part": "~effect/ai/Content/Part",
  },
  {
    type: "text-delta",
    id: "0",
    delta: " look at the",
    metadata: {},
    "~effect/ai/Content/Part": "~effect/ai/Content/Part",
  },
  {
    type: "text-delta",
    id: "0",
    delta: " package",
    metadata: {},
    "~effect/ai/Content/Part": "~effect/ai/Content/Part",
  },
  {
    type: "text-delta",
    id: "0",
    delta: ".json file to",
    metadata: {},
    "~effect/ai/Content/Part": "~effect/ai/Content/Part",
  },
  {
    type: "text-delta",
    id: "0",
    delta: " understan",
    metadata: {},
    "~effect/ai/Content/Part": "~effect/ai/Content/Part",
  },
  {
    type: "text-delta",
    id: "0",
    delta: "d the project",
    metadata: {},
    "~effect/ai/Content/Part": "~effect/ai/Content/Part",
  },
  {
    type: "text-delta",
    id: "0",
    delta: " structure and available",
    metadata: {},
    "~effect/ai/Content/Part": "~effect/ai/Content/Part",
  },
  {
    type: "text-delta",
    id: "0",
    delta: " l",
    metadata: {},
    "~effect/ai/Content/Part": "~effect/ai/Content/Part",
  },
  {
    type: "text-delta",
    id: "0",
    delta: "inting commands",
    metadata: {},
    "~effect/ai/Content/Part": "~effect/ai/Content/Part",
  },
  {
    type: "text-delta",
    id: "0",
    delta: ":",
    metadata: {},
    "~effect/ai/Content/Part": "~effect/ai/Content/Part",
  },
  {
    type: "text-end",
    id: "0",
    metadata: {},
    "~effect/ai/Content/Part": "~effect/ai/Content/Part",
  },
  {
    type: "tool-call",
    id: "toolu_01QxBQJ6yAK5SWXVa1sdyf65",
    name: "Tool2",
    params: {
      foo: "Bar",
    },
    providerExecuted: false,
    metadata: {},
    "~effect/ai/Content/Part": "~effect/ai/Content/Part",
  },
  {
    id: "toolu_01QxBQJ6yAK5SWXVa1sdyf65",
    type: "tool-result",
    isFailure: false,
    "~effect/ai/Content/Part": "~effect/ai/Content/Part",
    name: "Tool2",
    result: {
      foo: "bar baz",
    },
    encodedResult: {
      foo: "bar baz",
    },
    providerExecuted: false,
    metadata: {},
  },
];

const DERIVED = Prompt.fromResponseParts(PARTS as any);
console.log("DERIVED");
console.dir(DERIVED, { depth: null });

As you can see, the input parts are completely fine, and EVERY TOOL CALL is followed IMMEDIATELY BY A TOOL RESULT, which is correct, BUT the derived Prompt has TEXT CONTENT after the tool-call part, which causes the following error:

[16:19:17.135] DEBUG (#699): Fiber terminated with an unhandled error
  HttpResponseError: StatusCode: non 2xx status code (400 POST https://api.anthropic.com/v1/messages)

Bad Request - Check request parameters, headers, and body format against API documentation.

Response Body: {
  "type": "error",
  "error": {
    "type": "invalid_request_error",
    "message": "messages.2: `tool_use` ids were found without `tool_result` blocks immediately after: toolu_..., toolu_.... Each `tool_use` block must have a corresponding `tool_result` block in the next message."
  },
  "request_id": "req_..."
}
    at <anonymous> (/.../node_modules/@effect/ai/dist/esm/AiError.js:304:59)
  fiber: streamParts

There's a simple patch I have implemented locally, where we just make sure to sort the assistantParts, so that tool calls are always last, like they're supposed to be.

if (assistantParts.length > 0) {
    messages.push(makeMessage("assistant", {
      content: assistantParts.sort((a, b) => {
        // tool-call goes last
        if (a.type === "tool-call" && b.type !== "tool-call") return 1;
        if (a.type !== "tool-call" && b.type === "tool-call") return -1;
        return 0;
      })
    }));
  }

What is the expected behavior?

Expected to reduce the Prompt correctly, so that we have tool results IMMEDIATELY after tool calls

What do you see instead?

text parts after tool calls, causing Anthropic to error

Additional information

No response

Nov 03 '25 14:11 skoshx