spring-ai
spring-ai copied to clipboard
spring-projects
Added new VertexAI Anthropic's model module
This pull request introduces a new module to the Spring AI framework, designed to integrate and expand the use of Anthropic Claude LLM models within the VertexAI platform. The integration leverages the Model Garden feature of VertexAI to provide seamless and consistent access to Claude LLMs.
Hi @tzolov @markpollack just to understand if you could evaluate the merge of this PR that enables the use Anthropic's models from GCP Vertex Model Garden that has variations from both Anthropic standard endpoints but also from Vertex already implemented (Gemini) modules.
Thanks in advance!
Hi, sorry for letting this sit so long. This is a pretty big PR and i'm growing concerned over the number of models and variations each with different code bases. Just so verify my understanding with you, the need for this new API as compared to using the Vertex API to access the model is that the Anthropic model you want to use is different than the one exposed via Vertex API?
Hi, that's correct. Initially, I tried using the Vertex API to take advantage of Google’s SDK and also benefit from the authentication component. I had also seen the possibility of changing the publisher by dynamically specifying the model in the URL. Unfortunately, it’s not possible to use custom models that the Anthropic APIs within VertexAI’s model garden require (here’s the link with the details: https://console.cloud.google.com/vertex-ai/publishers/anthropic/model-garden/claude-3-5-sonnet?_ga=2.151856633.1780909182.1726601103-828101531.1719608013). This is the reason why I created this additional version.
Hi @alessiobertazzo88 could you please confirm the following, just to be sure that I understand this right?
You pick Sonnet 3.5 for example from the Model Garden in Vertex AI, Enable the API and would like to access it programmatically from Spring AI and a supporting SDK.
Hi @ddobrin Correct! The Google SDK now is only used for authentication because of lack of models for Anthropic use.
@alessiobertazzo88 thanks for the confirmation.
I am waiting for a small change to be made in Spring AI for another issue, which should help validate this
From google's site -"The Generative AI module in the Vertex AI SDK is deprecated and will be removed after June 24, 2026"
https://cloud.google.com/vertex-ai/generative-ai/docs/deprecations/genai-vertexai-sdk
@ddobrin @alessiobertazzo88 I suspect all new features we add should be on that stack, though Jun 2026 is a year away. Thoughts gents?
@alessiobertazzo88 @markpollack with the GenAI SDK replacing the Vertex SDK (which is on the subset path), we should any new functionality to the new stack.
New features as a result of model evolution are being added to the GenAI SDK at this time. For example: ThinkingBudget in Gemini Pro 2.5 is supported only in the new SDK.
The new SDK gives you both API Key and Google credentials based access to GCP.