pi-mono
pi-mono copied to clipboard
Anthropic's ephemeral cache markers are not used in OpenRouter API calls
The issue:
When I use Anthropic models via OpenRouter, apparently cache markers are not used, if I look at my OR activity details:
And there's no mention of caching in generation details (these are several requests from a single pi session):
Possible cause:
I asked Opus 4.5 to find why. Here are their ideas:
When using Anthropic models via OpenRouter (e.g., anthropic/claude-3.5-sonnet), the harness routes through openai-completions.js instead of anthropic.js. This means no cache_control: { type: "ephemeral" } markers are being added to the messages.
In openai-completions.js (used for OpenRouter): No cache_control markers are added anywhere