opentelemetry-python-contrib icon indicating copy to clipboard operation
opentelemetry-python-contrib copied to clipboard

opentelemetry-instrumentation-vertexai: support StreamGenerateContent

Open codefromthecrypt opened this issue 8 months ago • 2 comments

What problem do you want to solve?

Right now, if you run use langchain with VertexAI in a chatbot, the initialization will appear like this:

    return ChatVertexAI(
        model_name=os.getenv("CHAT_MODEL"), streaming=True, temperature=temperature
    )

When a generation occurs, if you have bootstrapped your deps, you'll see a normal platform span like this:

google.cloud.aiplatform.v1beta1.PredictionService/StreamGenerateContent

You won't yet see a genai span as while non-streaming has, streaming hasn't yet been implemented. https://github.com/open-telemetry/opentelemetry-python-contrib/blob/opentelemetry-instrumentation-vertexai%3D%3D2.0b0/instrumentation-genai/opentelemetry-instrumentation-vertexai/src/opentelemetry/instrumentation/vertexai/init.py#L81-L82

Describe the solution you'd like

I'd like the next release of opentelemetry-instrumentation-vertexai to include google.cloud.aiplatform.v1beta1.PredictionService/StreamGenerateContent

Describe alternatives you've considered

Currently, we use langtrace as the data is most similar to the semantic conventions.

    from langtrace_python_sdk.instrumentation import VertexAIInstrumentation

    VertexAIInstrumentation().instrument()
    return ChatVertexAI(
        model_name=os.getenv("CHAT_MODEL"), streaming=True, temperature=temperature
    )

Additional Context

cc @aabmass and FYI this is the specific code I would like to remove https://github.com/elastic/elasticsearch-labs/blob/main/example-apps/chatbot-rag-app/api/llm_integrations.py#L23-L26

Would you like to implement a fix?

None

codefromthecrypt avatar Feb 25 '25 07:02 codefromthecrypt