ai
ai copied to clipboard
Update typing for Amazon Bedrock Provider Options to include cache points
Description
Setting cache control breakpoints is available for the amazon bedrock provider, as shown here: https://ai-sdk.dev/providers/ai-sdk-providers/amazon-bedrock#cache-points
The method is by setting the cachePoint property on a message via providerOptions
const result = await generateText({
model: bedrock('anthropic.claude-3-5-sonnet-20241022-v2:0'),
messages: [
{
role: 'system',
content: `You are an expert on William Gibson's cyberpunk literature and themes. You have access to the following academic analysis: ${cyberpunkAnalysis}`,
providerOptions: {
bedrock: { cachePoint: { type: 'default' } },
},
},
{
role: 'user',
content:
'What are the key cyberpunk themes that Gibson explores in Neuromancer?',
},
],
});
However, the actual BedrockProviderOptions type does not contain the cachePoint field:
In contrast, you can look at the AnthropicProviderOptions, which does contain cache control fields
This is pretty helpful for just being able to set a type on the provider options, and I would appreciate if it was included in the SDK by default.
AI SDK Version
"[email protected]" "@ai-sdk/[email protected]"
Code of Conduct
- [x] I agree to follow this project's Code of Conduct