feat: [ENG-3645] AI Gateway Reasoning Support - Anthropic

[connortbot]

• Scaffolds out broad reasoning configuration support on the AI Gateway by updating both input and output to handle reasoning summary, config, etc.
• Adds mapping support for using Anthropic thinking

Other notable fixes

• Fix wrong openrouter haiku 4.5 slug
• Fix incorrectly adding stream usage to responses API input, when it should've been in the responsers->completions transform
• Update streamParser to consolidate reasoning deltas

[vercel[bot]]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Preview	Comments	Updated (UTC)
helicone	Ready	Preview	Comment	Nov 27, 2025 0:31am
helicone-bifrost	Ready	Preview	Comment	Nov 27, 2025 0:31am
helicone-eu	Ready	Preview	Comment	Nov 27, 2025 0:31am

[claude[bot]]

Claude finished @connortbot's task —— View job

---

PR Review: AI Gateway Reasoning Support - Anthropic

Todo List:

• [x] Analyze core transformation logic in request/response mappers
• [x] Review stream parser changes for data handling issues
• [x] Check type safety and API consistency
• [x] Evaluate security implications and performance
• [x] Provide final score and recommendations

Review Analysis

Critical Issues Found

1. Silent Temperature Override (Critical)

Location: packages/llm-mapper/transform/providers/openai/request/toAnthropic.ts:49

// temperature 1 is required for Anthropic models to enable reasoning
antBody.temperature = 1;

Issue: This unconditionally overwrites any user-provided temperature when reasoning is enabled. If a user sets temperature: 0.5 with reasoning, it gets silently changed to 1, potentially breaking their expected model behavior.

Suggested Fix:

if (antBody.temperature !== undefined && antBody.temperature !== 1) {
  console.warn(`Temperature changed from ${antBody.temperature} to 1 for reasoning compatibility`);
}
antBody.temperature = 1;

2. Data Loss in Stream Consolidation (Critical)

Location: valhalla/jawn/src/utils/streamParser.ts:53-56

reasoning_details:
  c.delta.reasoning_details && cur.choices[i].delta.reasoning_details
    ? [...c.delta.reasoning_details, ...cur.choices[i].delta.reasoning_details]
    : c.delta.reasoning_details || cur.choices[i].delta.reasoning_details,

Issue: The logic is inconsistent with reasoning field handling. It properly merges arrays when BOTH chunks have reasoning_details, but uses OR fallback otherwise. This could lose data if multiple non-overlapping chunks contain reasoning_details.

Suggested Fix:

reasoning_details: cur.choices[i].delta.reasoning_details
  ? [...(c.delta.reasoning_details || []), ...cur.choices[i].delta.reasoning_details]
  : c.delta.reasoning_details,

Code Quality Issues

3. Type Safety Violations

Location: packages/llm-mapper/transform/providers/openai/request/toAnthropic.ts:210

throw new Error(`Unsupported content type: ${(part as any).type}`);

Issue: Using any type defeats TypeScript's type safety. Consider using proper union types or type guards.

4. Inconsistent Error Handling

Location: worker/src/lib/ai-gateway/SimpleAIGateway.ts:428-430

private requiresReasoningOptions(providerModelId: string): boolean {
  return providerModelId.includes("claude");
}

Issue: String-based provider detection is fragile. Consider using enum or more robust pattern matching.

Positive Aspects

✅ Comprehensive Type Definitions: Good addition of reasoning types across the codebase
✅ Proper Stream Handling: Thinking deltas are correctly accumulated and mapped
✅ Multi-turn Support: Signature handling for conversation continuity
✅ Validation Logic: Required field validation for reasoning options
✅ Clean API Mapping: Bidirectional conversion between OpenAI and Anthropic formats

Security & Performance

🔒 Security: No security issues identified - proper input validation and sanitization
⚡ Performance: Stream processing is efficient, no obvious performance bottlenecks

Minor Issues

• Missing JSDoc comments for complex transformation logic
• Could benefit from unit tests for edge cases in stream consolidation
• Consider adding configuration for temperature override behavior

---

Score: 6/10 - Minor issues with critical data handling bugs

Suggestions Summary:

• CRITICAL: Fix temperature override to preserve user intent or provide clear warning
• CRITICAL: Fix reasoning_details consolidation logic to prevent data loss
• IMPROVEMENT: Replace any types with proper type guards
• IMPROVEMENT: Make provider detection more robust
• ENHANCEMENT: Add unit tests for stream consolidation edge cases

The implementation is solid overall with good architecture, but the two critical data handling issues need to be addressed before merging.

[greptile-apps[bot]]

Greptile Overview

Greptile Summary

This PR implements reasoning support for Anthropic's thinking feature on the AI Gateway. It adds bidirectional mapping between OpenAI's reasoning format and Anthropic's thinking blocks, enabling users to leverage Anthropic's extended thinking capabilities through a unified interface.

Key Changes:

• Maps reasoning_effort + reasoning_options.budget_tokens to Anthropic's thinking configuration
• Extracts thinking blocks from Anthropic responses as reasoning and reasoning_details in OpenAI format
• Handles streaming thinking deltas and signature accumulation for multi-turn conversations
• Adds validation and type definitions across request/response formats
• Strips reasoning_options for non-Claude models to prevent API errors
• Fixes OpenRouter model ID for Claude 4.5 Haiku

Issues Found:

• Temperature override silently replaces user-provided values when reasoning is enabled
• Stream consolidation logic discards previous reasoning_details chunks, causing data loss

Confidence Score: 3/5

• Safe to merge with caution - has logical issues that could cause data loss and unexpected behavior
• Two critical logic issues identified: (1) temperature override silently discards user preferences, potentially breaking expected model behavior, and (2) stream consolidation bug causes data loss in reasoning_details, though this may only affect specific multi-chunk scenarios. The rest of the implementation appears solid with proper type safety and comprehensive mapping logic.
• Pay close attention to valhalla/jawn/src/utils/streamParser.ts (data loss bug) and packages/llm-mapper/transform/providers/openai/request/toAnthropic.ts (silent parameter override)

Important Files Changed

File Analysis

Filename	Score	Overview
packages/llm-mapper/transform/providers/openai/request/toAnthropic.ts	4/5	Maps OpenAI reasoning config to Anthropic thinking blocks with required validation and temperature override
packages/llm-mapper/transform/providers/anthropic/streamedResponse/toOpenai.ts	5/5	Streams thinking deltas as reasoning, accumulates signatures for multi-turn support
worker/src/lib/ai-gateway/SimpleAIGateway.ts	4/5	Strips reasoning_options for non-Claude models, retains for Anthropic provider
valhalla/jawn/src/utils/streamParser.ts	5/5	Consolidates reasoning and reasoning_details fields in streamed responses

Sequence Diagram

sequenceDiagram
    participant Client
    participant Gateway as AI Gateway
    participant Mapper as LLM Mapper
    participant Anthropic
    
    Client->>Gateway: POST /chat/completions<br/>{reasoning_effort, reasoning_options}
    Gateway->>Gateway: Validate request
    Gateway->>Mapper: toAnthropic()
    Mapper->>Mapper: Map reasoning_options.budget_tokens<br/>to thinking.budget_tokens
    Mapper->>Mapper: Set temperature = 1<br/>(required for thinking)
    Mapper->>Mapper: Convert messages<br/>(thinking blocks first)
    Mapper->>Anthropic: POST /messages<br/>{thinking: {budget_tokens}}
    
    alt Streaming Response
        Anthropic-->>Mapper: thinking_delta events
        Mapper->>Mapper: Accumulate thinking + signature
        Mapper-->>Gateway: Stream reasoning deltas
        Gateway-->>Client: Stream with reasoning field
        Anthropic-->>Mapper: message_delta (final)
        Mapper->>Mapper: Collect reasoning_details<br/>from accumulated state
        Mapper-->>Gateway: Final chunk with reasoning_details
        Gateway-->>Client: Complete stream
    else Non-Streaming Response
        Anthropic-->>Mapper: Response with thinking blocks
        Mapper->>Mapper: Extract thinking blocks
        Mapper->>Mapper: Map to reasoning + reasoning_details
        Mapper-->>Gateway: OpenAI format response
        Gateway-->>Client: Response with reasoning fields
    end