Implement Tools Config Caching for Anthropic Claude

[chezsmithy]

Validations

• [x] I believe this is a way to improve. I'll try to join the Continue Discord for questions
• [x] I'm not able to find an open issue that requests the same enhancement

Problem

I would like to add prompt caching for the tools configuration for Anthropic, AWS Bedrock and Vertex hosted Claude models. Tools config is highly repeatable and caching this configuration limits the number of tokens used. I've validated this with my local configuration using AWS Bedrock with Claude 3.7.

For AWS Bedrock and Claude 3.7 see this page for more information: https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-caching.html

Solution

I've identified through local testing that it's possible to add caching support for tools config. This allows the config to enter the prompt cache and then significantly lowers the number of tokens consumed on each subsequent call to the LLM.

I've identified that AWS Bedrock, Anthropic and Vertex seem to all support this feature tho in slightly different ways.

Here is what I would propose as PRs.

1. Create a new configuration item to enable toolsConfigCache under the cacheBehavior key.
2. Using that key have contributors with access to each back end independently add the caching support as a separate PR with updated tests.

Implementation Questions

1. Should I implement the configuration changes in JSON and YAML?
2. Should the configuration only be enabled for certain models and providers in a specific way?