Bedrock streaming mode fails to throw exceptions correctly

[horochx]

What Happened?

params：

{
  "model": "us.anthropic.claude-3-7-sonnet-20250219-v1:0",
  "messages": [
    {
      "role": "system",
      "content": [
        {
          "type": "text",
          "text": "You should always follow the instructions and output a valid JSON object.\nThe structure of the JSON object you can found in the instructions, use {\"answer\": \"$your_answer\"} as the default structure\nif you are not sure about the structure.\n\n<instructions>\nTell a short programmer joke. Return it in the result field of a JSON.\n</instructions>",
          "cache_control": { "type": "ephemeral" }
        }
      ]
    },
    { "role": "assistant", "content": "\n```JSON" }
  ],
  "stop": ["```\n", "\n```"],
  "stream": true,
  "thinking": { "type": "enabled", "budget_tokens": 1024 }
}

bedrock response: status code 200，json:

{
  message:
    "The model returned the following errors: messages.1.content.0.type: Expected `thinking` or `redacted_thinking`, but found `text`. When `thinking` is enabled, a final `assistant` message must start with a thinking block. We recommend you include thinking blocks from previous turns. To avoid this requirement, disable `thinking`. Please consult our documentation at https://docs.anthropic.com/en/docs/build-with-claude/extended-thinking",
}

portkey response:

{
  "id": "bedrock-1750128201253",
  "object": "chat.completion.chunk",
  "created": 1750128201,
  "model": "us.anthropic.claude-3-7-sonnet-20250219-v1:0",
  "provider": "bedrock",
  "choices": [{ "index": 0, "delta": { "role": "assistant" } }]
}

What Should Have Happened?

{
  "error": {
    "message": "bedrock error: The model returned the following errors: messages.1.content.0.type: Expected `thinking` or `redacted_thinking`, but found `text`. When `thinking` is enabled, a final `assistant` message must start with a thinking block. We recommend you include thinking blocks from previous turns. To avoid this requirement, disable `thinking`. Please consult our documentation at https://docs.anthropic.com/en/docs/build-with-claude/extended-thinking",
    "type": null,
    "param": null,
    "code": null
  },
  "provider": "bedrock"
}

Relevant Code Snippet

No response

Your Twitter/LinkedIn

No response

[narengogi]

@horochx openai streaming has no concept of an error chunk maybe we can wait for the first chunk before establishing a stream connection and return a failed http code

[jczstudios]

Duplicate of #1047