Error using dspy with Google Vertex

[onel]

I tried to use DSPy with vertex and I got this error when running a script.

Traceback (most recent call last):
  File "/app/./utils/test.py", line 10, in <module>
    vertex_ai = GoogleVertexAI(
                ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/dsp/modules/google_vertex_ai.py", line 60, in __init__
    self._init_vertexai(kwargs)
  File "/usr/local/lib/python3.12/site-packages/dsp/modules/google_vertex_ai.py", line 114, in _init_vertexai
    vertexai.init(
    ^^^^^^^^
NameError: name 'vertexai' is not defined

Code

from dsp.modules import GoogleVertexAI

vertex_ai = GoogleVertexAI(
    model_name="claude-3-5-sonnet-v2@20241022",
    location="europe-west1",
    project="",
    credentials="google-cloud-creds.json"
)
dspy.configure(lm=vertex_ai)

requirements.txt

dspy-ai
dspy-ai[google-vertex-ai]

Is there an official documentation for how to use Vertex with DSPy? I saw the PR where this was added but nothing on the main docs site.

[okhat]

please use dspy.LM see the landing page dspy.ai for how to set it up

[onel]

Thanks for the quick response, but dspy.LM goes straight to Anthropic. We have a project requirement to use Vertex for model calling. Anyway to configure dspy.LM to call google, or a certain region?

[arnavsinghvi11]

Hi @onel,

dspy.LM uses LiteLLM so you can set it as you would for the Google models.

something like this dspy.LM("model_name", vertex_credentials=...)

[onel]

Thanks for the info. I'll go through the lite llm docs.

Is there any page that explains all of this on dspy.ai?

Are the docs being rewritten ATM? I see a lot of doc links are broken and pages missing.

[onel]

I hope the question didn't come off as weird. I asked because I'd love to help with documentation if you guys need help.

[okhat]

Thank you @onel ! Basically for vertex, you can do:

dspy.LM("vertex_ai/gemini-pro",  vertex_credentials=vertex_credentials_json)