Support Anthropic prompt caching for tool definitions

[namukang]

Feature Description

The AI SDK currently supports Anthropic's prompt caching for system messages and messages (including tool calls and tool results), but not tool definitions: https://github.com/vercel/ai/discussions/3062#discussioncomment-10702467

Anthropic docs: https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching#what-can-be-cached AI SDK docs: https://sdk.vercel.ai/providers/ai-sdk-providers/anthropic#cache-control

I'd like to be able to implement prompt caching on the tool definitions using the AI SDK. Thank you!

[mayrsascha]

@lgrammel, any ETA on this one? We're hitting the 40K input token limit/minute, and since cache reads don't count against that limit, having this would help quite a lot.

[lgrammel]

It's on the roadmap, expected in v5.0 or v5.1

[Lustyn]

This shouldn't be a major blocker to getting good caching, since putting cache control on the last tool is effectively equivalent to putting cache control on a zero-size system block inserted at the start.

The only use-case that can't be served by caching early or late in the system prompt is creating a cache point in the middle of the tool list, which is likely to result in poor cache performance anyways. That would imply you have tool definitions changing request-to-request which will prevent system and message sections from (realistically) ever caching.

[cloudycotton]

@lgrammel was this fixed in the v5.0? I don't see us hitting cache at all.