dspy icon indicating copy to clipboard operation
dspy copied to clipboard

feat(dsp): Vertex Anthropic Module

Open LukvonStrom opened this issue 6 months ago • 1 comments

📝 Changes Description

This MR/PR contains the following changes:

  • Add a seperate module in dsp for Anthropic via VertexAI as described here: https://cloud.google.com/vertex-ai/generative-ai/docs/partner-models/use-claude#use_the_anthropic_sdk

This is just a rough PR to gauge interest and discuss whether the existing Anthropic Module can be made generic enough to support Vertex as well (atm it shares like 95% code).

Suprisingly despite claiming it, the existing vertex module does not support anthropic in my local setup (the module is in fact not really well documented). ...

✅ Contributor Checklist

  • [x] Pre-Commit checks are passing (locally and remotely)
  • [x] Title of your PR / MR corresponds to the required format
  • [x] Commit message follows required format {label}(dspy): {message}

⚠️ Warnings

Anything we should be aware of ?

I am using this in my setup with claude-3-haiku@20240307 and it works. I cannot guarantee that for any other claude model at the moment.

The reproduction code for instantiation looks like this:

from dsp.modules import ClaudeVertex
llm = ClaudeVertex(
    model="claude-3-haiku@20240307",
    project="(snip)",
    location="europe-west1",
    temperature=0,
)

There has been some weird behaviour when passing in the Serviceaccount credentials like this:

path_to_sa_file = "account.json"

os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = path_to_sa_file
from dsp.modules import ClaudeVertex
from google.oauth2 import service_account

credentials_sa = service_account.Credentials.from_service_account_file(path_to_sa_file)
gpt35 = ClaudeVertex(
    model="claude-3-haiku@20240307",
    project="(snip)",
    location="europe-west1",
    credentials=credentials_sa,
    temperature=0,
)

Setting it via default credentials in the gcloud cli helped.

LukvonStrom avatar Aug 06 '24 15:08 LukvonStrom