Adding support for vextex and gemini

[PierreColombo]

Hello,

I see that you currently support clients like Anthropic, OpenAI, and Bedrock. We would love to see Vertex AI added as a supported client. Google has made a strong comeback in the LLM space with a leading model that excels in many use cases, including ours.

Do you think this could be made possible?

Best regards, Pierre

[alexmojaki]

Experimenting with this, this code:

import vertexai
from vertexai.generative_models import GenerativeModel

import logfire

logfire.configure(scrubbing=False)
vertexai.init()


original_generate_content = GenerativeModel._generate_content


def instrumented_generate_content(self: GenerativeModel, contents, *args, **kwargs):
    with logfire.span('generate_content', request=contents) as span:
        result = original_generate_content(self, contents, *args, **kwargs)
        span.set_attribute('response', result)
        return result


GenerativeModel._generate_content = instrumented_generate_content

model = GenerativeModel('gemini-1.5-flash-002')

chat = model.start_chat()
chat.send_message('What is 2+2?')
chat.send_message('really?')

on the branch of https://github.com/pydantic/logfire/pull/799

produces:

{
  "request": [
    {
      "role": "user",
      "parts": [
        {
          "text": "What is 2+2?"
        }
      ]
    },
    {
      "role": "model",
      "parts": [
        {
          "text": "2 + 2 = 4\n"
        }
      ]
    },
    {
      "role": "user",
      "parts": [
        {
          "text": "really?"
        }
      ]
    }
  ],
  "response": {
    "candidates": [
      {
        "content": {
          "role": "model",
          "parts": [
            {
              "text": "Yes, really.  2 + 2 = 4 is a fundamental fact of arithmetic.\n"
            }
          ]
        },
        "finish_reason": "STOP",
        "avg_logprobs": -0.08722147345542908
      }
    ],
    "usage_metadata": {
      "prompt_token_count": 17,
      "candidates_token_count": 20,
      "total_token_count": 37
    },
    "model_version": "gemini-1.5-flash-002"
  }
}

[0xRaduan]

Any update on this, @alexmojaki? Came here from the docs, and don't see a native support of gemini for logfire, only through pydantic ai.

[alexmojaki]

Try using https://github.com/open-telemetry/opentelemetry-python-contrib/tree/main/instrumentation-genai/opentelemetry-instrumentation-vertexai

[0xRaduan]

Okay I spent some time looking and it was just easier for me to just wrap gemini into openai client. I probably lost some gemini specific functionality because of this, but it's good enough for my scrappy use case so far.

api_key = os.getenv("GOOGLE_API_KEY")
if not api_key:
    raise ValueError("GOOGLE_API_KEY environment variable not set")

client = OpenAI(
    api_key=api_key, base_url="https://generativelanguage.googleapis.com/v1beta/"
)
logfire.instrument_openai(client)

Based on this blog post: https://developers.googleblog.com/en/gemini-is-now-accessible-from-the-openai-library/

Note: the UI on this is a bit broken though, but I am personally fine with that for now. Would be great if that was supported out of the box.

[alexmojaki]

OK so I think we need some docs about this giving some options:

1. OTel libraries:
   • https://github.com/open-telemetry/opentelemetry-python-contrib/tree/main/instrumentation-genai/opentelemetry-instrumentation-vertexai
   • https://github.com/open-telemetry/opentelemetry-python-contrib/tree/main/instrumentation-genai/opentelemetry-instrumentation-google-genai
2. instrument_openai like suggested (maybe this also works with things like OpenRouter)
3. PydanticAI
4. Link to this issue for tracking and other ideas like https://github.com/pydantic/logfire/issues/788#issuecomment-2590815656

[0xRaduan]

Oh, @alexmojaki - can I use pydantic ai as a non agent framework just for structured output purposes?

I am currently using instructor, but happy to migrate if you guys support just structured output use cases.

Also, feel free to point me to the right direction of where to put this docs, and I am happy to contribute.

[samuelcolvin]

Yes you can.

Currently we handle structured outputs via tool calls, but as per https://github.com/pydantic/pydantic-ai/issues/582 we'll soon support other mechanisms (e.g. builtin mechanisms in openai and gemini, and JSON Schema prompt for other providers).

https://ai.pydantic.dev/results/#structured-result-validation has some good docs.

[alexmojaki]

Also, feel free to point me to the right direction of where to put this docs, and I am happy to contribute.

That'd be great. See https://logfire.pydantic.dev/docs/integrations/llms/llamaindex/ as an example of similar docs. The source is docs/integrations/llms/llamaindex.md.

[sandeep540]

Use this for new SDK

https://github.com/open-telemetry/opentelemetry-python-contrib/tree/main/instrumentation-genai/opentelemetry-instrumentation-google-genai

[smazurov]

adding

from opentelemetry.instrumentation.google_genai import GoogleGenAiSdkInstrumentor
from google.genai import Client

GoogleGenAiSdkInstrumentor().instrument()

Produces some extra info, but it doesn't quite correctly decorate the spans. I'm not seeing token usage or prompts/responses as first class citizens in the UI.

Here's the raw attribute examples:

{
  "gen_ai.completion.0.content": "{\n  \"department\": \"Engineering\",\n  \"reason\": \"This an example of a response.\"\n}",
  "gen_ai.completion.0.role": "assistant",
  "gen_ai.request.model": "gemini-2.5-pro-preview-03-25",
  "gen_ai.response.model": "gemini-2.5-pro-preview-03-25",
  "gen_ai.system": "Gemini",
  "gen_ai.usage.completion_tokens": 167,
  "gen_ai.usage.prompt_tokens": 8678,
  "llm.request.type": "completion",
  "llm.usage.total_tokens": 10400
}

Is there a way to bridge the gap until logfire has its own wrapper?

[alexmojaki]

OK I've released a new method logfire.instrument_google_genai() in https://github.com/pydantic/logfire/pull/1217. It uses https://github.com/open-telemetry/opentelemetry-python-contrib/tree/main/instrumentation-genai/opentelemetry-instrumentation-google-genai and requires installing opentelemetry-instrumentation-google-genai. If you already have that installed, I suggest upgrading it. I see that the package has some issues, so for now this is experimental, please try it and give feedback to help prioritize this. Also you will probably want to set the env var OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true.

[alexmojaki]

In case you already tried, sorry the logfire SDK release hadn't actually completed yet, now it has.

BTW another good thing to try is logfire.instrument_httpx(capture_all=True).

[cvaisnor]

Just tried the above approach. I am trying to get the token cost information working on Logfire for Gemini-2.5-flash. It currently says UNKNOWN.

However, I noticed that there are two libraries indicating the same functionality. I have provided both of their PyPi links below. The library with the full 'generativeai' name is much more popular and updated two days ago, but that is NOT the library suggested to use in this fix.

• https://pypi.org/project/opentelemetry-instrumentation-google-generativeai/
• https://pypi.org/project/opentelemetry-instrumentation-google-genai/

I tried the above fix using the shorted-name library including adding logfire.instrument_httpx(capture_all=True) as well as setting the .env variable OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true but was not able to see any cost metrics in Logfire.

[alexmojaki]

Our pricing calculation is based on https://www.helicone.ai/llm-cost, which doesn't seem to have gemini-2.5-flash yet. I've opened https://github.com/Helicone/helicone/pull/4048. In the meantime, you can still see tokens and other stuff, right? Just not costs?

I tried https://pypi.org/project/opentelemetry-instrumentation-google-generativeai/ and it doesn't seem to work at all, did any part of it work for you?

[alexmojaki]

gemini-2.5-flash prices should show now

[aniketk13]

Hey @alexmojaki I am using a custom model Is there a way I can custom add the pricing for my model?

[anthony2261]

Trying to use logfire.instrument_google_genai() in my code, but when I try to make an api call, I get this error

    response = await client.aio.models.generate_content(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.11/site-packages/opentelemetry/instrumentation/google_genai/generate_content.py", line 680, in instrumented_generate_content
    helper.process_request(contents, config)
  File "/.venv/lib/python3.11/site-packages/opentelemetry/instrumentation/google_genai/generate_content.py", line 290, in process_request
    _add_request_options_to_span(
  File "/.venv/lib/python3.11/site-packages/opentelemetry/instrumentation/google_genai/generate_content.py", line 162, in _add_request_options_to_span
    attributes = flatten_dict(
                 ^^^^^^^^^^^^^
  File "/.venv/lib/python3.11/site-packages/opentelemetry/instrumentation/google_genai/dict_util.py", line 295, in flatten_dict
    return _flatten_dict(
           ^^^^^^^^^^^^^^
  File "/.venv/lib/python3.11/site-packages/opentelemetry/instrumentation/google_genai/dict_util.py", line 251, in _flatten_dict
    flattened = _flatten_value(
                ^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.11/site-packages/opentelemetry/instrumentation/google_genai/dict_util.py", line 228, in _flatten_value
    return _flatten_compound_value(
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.11/site-packages/opentelemetry/instrumentation/google_genai/dict_util.py", line 193, in _flatten_compound_value
    value.model_dump(),
    ^^^^^^^^^^^^^^^^^^
TypeError: BaseModel.model_dump() missing 1 required positional argument: 'self'

Using logfire[fastapi] 4.0.0 and opentelemetry-instrumentation-google-genai 0.3b0

[alexmojaki]

@anthony2261 please can you share example code? This works for me:

import asyncio
import os

from google.genai import Client, types

import logfire

os.environ['OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT'] = 'true'

logfire.configure()
logfire.instrument_google_genai()


def get_current_weather(location: str) -> str:
    """Returns the current weather."""
    return 'rainy'


async def main():
    client = Client()
    response = await client.aio.models.generate_content(
        model='gemini-2.5-flash',
        contents='What is the weather like in Boston?',
        config=types.GenerateContentConfig(tools=[get_current_weather]),
    )
    print(response)


asyncio.run(main())

[anthony2261]

Thanks @alexmojaki. Your code snippet worked for me, but not my code, so here's something reproducible you can try:

import asyncio
import os

import logfire
from google.genai import Client, types
from pydantic import BaseModel

os.environ["OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT"] = "true"

logfire.configure()
logfire.instrument_google_genai()

client = Client()


class ResponseData(BaseModel):
    """This is the data structure for the response."""
    answer: str


def main():
    response = await client.aio.models.generate_content(
        model="gemini-2.5-pro-preview-06-05",
        contents="Say hello to the user",
        config=types.GenerateContentConfig(
            response_mime_type="application/json",
            response_schema=ResponseData,
            thinking_config=types.ThinkingConfig(thinking_budget=128),
        ),
    )
    return response.parsed

asyncio.run(main())

My hunch is that response_schema is causing this.

[alexmojaki]

OK thanks, I see an existing issue: https://github.com/open-telemetry/opentelemetry-python-contrib/issues/3596

[anthony2261]

Ah perfect, and thanks for providing them with an example! I'll keep an eye out on the other issue then 👍

[alexmojaki]

I've implemented a workaround for the response_schema issue in https://github.com/pydantic/logfire/pull/1342 and released, so if you upgrade the SDK it should work.

[anthony2261]

Thanks @alexmojaki . FYI I solved this for my use case by forking the open-telemetry python contrib repo and applying this fix https://github.com/open-telemetry/opentelemetry-python-contrib/commit/69e1ba57660a644492fce9e5ca9084c1c58c5ff3 , allows me to filter values in logfire, rather than having the value as string (which is what safe_repr would do)

In case this helps the others