spring-ai
spring-ai copied to clipboard
spring-projects
Added suport to anthropic prompt cache
Thank you for taking time to contribute this pull request! You might have already read the [contributor guide][1], but as a reminder, please make sure to:
issue https://github.com/spring-projects/spring-ai/issues/1403
I added support to user prompt use cache, added test only to api because Chatmodel/AnthropicChatOptions not can set header beta to works cache
- Sign the contributor license agreement
- Rebase your changes on the latest
mainbranch and squash your commits - Add/Update unit tests as needed
- Run a build and make sure all tests pass prior to submission
thanks, can you explain the comment "Chatmodel/AnthropicChatOptions not can set header beta to works cache" please?
Anthropic models added caching as a beta feature that requires a custom header to be used. I couldn't find a way to send a custom header using the options in the Chatmodel/AnthropicChatOptions classes.
That's why I added integration tests only in the AnthropicApiIT.java class.
It would be nice if in the future there was an option to send a custom header in ChatModel class to test beta features like langchain4j https://github.com/langchain4j/langchain4j/issues/1591#issuecomment-2357792634.
With this type of option it would be easier to add integration tests for beta features in the AnthropicChatModelIT.java class.
Hi. Thanks for the clarification. I think it is a bit too much to get into M3 only because it modifies the AbstractMessage class and I'd like to think through the implications. I guess much like function calling, this is going to "table stakes" for leading model providers. I'll schedule it for M4 with the hopes we have a 5-ish week release cadence on our march to GA.
thanks for the nice PR.
@Claudio-code @markpollack @tzolov any chance this be merged any sooner than March?
We are waiting for prompt cache feature too. It should help us to save tokens. Our app sends screenshots to Anthropic and we chose Spring AI framework to deal with LLMs
I saw that the cache left beta, now I can add more complete tests to validate the creation of token caches and the reading of these caches.
Thanks, just going through the issues, will aim to merge for m7. thanks for you patience.
@Claudio-code , thanks for your patience. I've rebased your PR with recent changes, cherry-picking some parts and creating a new updated PR: https://github.com/spring-projects/spring-ai/pull/2669 You remain the author. Could you please review to ensure nothing was broken during this process?
