dspy
dspy copied to clipboard
feat(dsp): Vertex Anthropic Module
📝 Changes Description
This MR/PR contains the following changes:
- Add a seperate module in
dsp
for Anthropic via VertexAI as described here: https://cloud.google.com/vertex-ai/generative-ai/docs/partner-models/use-claude#use_the_anthropic_sdk
This is just a rough PR to gauge interest and discuss whether the existing Anthropic Module can be made generic enough to support Vertex as well (atm it shares like 95% code).
Suprisingly despite claiming it, the existing vertex module does not support anthropic in my local setup (the module is in fact not really well documented). ...
✅ Contributor Checklist
- [x] Pre-Commit checks are passing (locally and remotely)
- [x] Title of your PR / MR corresponds to the required format
- [x] Commit message follows required format {label}(dspy): {message}
⚠️ Warnings
Anything we should be aware of ?
I am using this in my setup with claude-3-haiku@20240307
and it works. I cannot guarantee that for any other claude model at the moment.
The reproduction code for instantiation looks like this:
from dsp.modules import ClaudeVertex
llm = ClaudeVertex(
model="claude-3-haiku@20240307",
project="(snip)",
location="europe-west1",
temperature=0,
)
There has been some weird behaviour when passing in the Serviceaccount credentials like this:
path_to_sa_file = "account.json"
os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = path_to_sa_file
from dsp.modules import ClaudeVertex
from google.oauth2 import service_account
credentials_sa = service_account.Credentials.from_service_account_file(path_to_sa_file)
gpt35 = ClaudeVertex(
model="claude-3-haiku@20240307",
project="(snip)",
location="europe-west1",
credentials=credentials_sa,
temperature=0,
)
Setting it via default credentials in the gcloud cli helped.