PromptingTools.jl
PromptingTools.jl copied to clipboard
Implement caching support for Anthropic API
Implement caching support for Anthropic API
This pull request proposes enhancements to the PromptingTools.jl package by introducing caching support for the Anthropic API wrapper. Caching allows for optimized API usage by minimizing repeated requests and reducing computational load.
Key Changes:
- Added
cache
argument toaigenerate
function for specifying cache state. - Integrated helper functions for caching:
_generate_cache_key
,_get_from_cache
, and_add_to_cache
. - Updated
AIMessage
with ameta
field to store cache performance statistics. - Initialized global caches:
SYSTEM_CACHE
,TOOLS_CACHE
,LAST_CACHE
.
Files Modified:
-
llm_anthropic.jl
-
messages.jl
Benefits:
- Reduces redundant API requests, enhancing efficiency and performance.