ai-sdk-provider
ai-sdk-provider copied to clipboard
Caching doesn't work with tool use
I think caching is broken with tool use. What I've observed is that when the arguments is serialized in a different order than previous messages, there is no cache hit.
I observed this with Anthropic / Claude 4.
The serialization seems to be happening here: https://github.com/OpenRouterTeam/ai-sdk-provider/blob/b23f2d580dc0688e5af1124e68c0e98b892e58fb/src/chat/convert-to-openrouter-chat-messages.ts#L179
JSON is an unordered sets of name/value pairs.
Sending { input: { a: 1, b: 2 } } after having cached { input: { b: 2, a: 1 } } should give a cache hit.