openinference icon indicating copy to clipboard operation
openinference copied to clipboard

[bug] Tool Results Not Displayed in Phoenix UI with Pydantic AI

Open luzmkk opened this issue 1 month ago • 1 comments

Describe the bug

Tool results from Pydantic AI agents are not displayed in the Phoenix UI when using OpenInferenceSpanProcessor with Pydantic AI instrumentation. While tool calls are visible with their arguments, the corresponding tool responses/results are missing from both the LLM span and individual tool call spans in the Phoenix UI.

To Reproduce

Steps to reproduce the behavior:

  1. Set up Pydantic AI with OpenTelemetry instrumentation:
from openinference.instrumentation.pydantic_ai import OpenInferenceSpanProcessor
from opentelemetry import trace
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from pydantic_ai import Agent
from pydantic_ai.models.instrumented import InstrumentationSettings

tracer_provider = TracerProvider()
trace.set_tracer_provider(tracer_provider)

# Add OpenInferenceSpanProcessor
tracer_provider.add_span_processor(OpenInferenceSpanProcessor())

# Add OTLP exporter to Phoenix
endpoint = "http://phoenix:6006/v1/traces"
exporter = OTLPSpanExporter(endpoint=endpoint)
tracer_provider.add_span_processor(BatchSpanProcessor(exporter))

# Instrument with version 2
Agent.instrument_all(InstrumentationSettings(version=2))  # same with `version=3`
  1. Create and run a Pydantic AI agent with function tools
  2. Execute the agent to trigger tool calls
  3. View the traces in Phoenix UI at http://localhost:6006

Expected behavior

  • Tool results should be visible in the LLM span showing the complete tool call flow (request + response)
  • Individual tool call spans should show both input arguments and output results
  • Tool responses should be displayed in a readable format (either Markdown or Text view)

Screenshots

  1. LLM Span - Missing Tool Result Image

  2. Tool Call Span - Missing Tool Result Image

  3. React Error when switching from Markdown to Text Image

Desktop (please complete the following information):

  • OS: macOS
  • Python Version: 3.12
  • pydantic-ai Version: 1.18.0
  • openinference-instrumentation-pydantic-ai Version: 0.1.9
  • opentelemetry-sdk Version: 1.29.0
  • Phoenix Version: Latest (Docker image: arizephoenix/phoenix)

Additional context

  • When OpenInferenceSpanProcessor is commented out (direct OTLP export without transformation), tool results ARE present in the raw span attributes:

    • Version 2: tool_response attribute contains the tool output
    • Version 3: gen_ai.tool.call.result attribute contains the tool output
  • This suggests the tool results are being emitted by Pydantic AI instrumentation but are not being properly transformed or preserved by OpenInferenceSpanProcessor

  • The processor appears to transform tool arguments correctly but loses or fails to transform tool results/responses

  • Tested with both version=2 and version=3 of InstrumentationSettings - same issue occurs with both versions

  • Package versions:

pydantic-ai==1.18.0
openinference-instrumentation-pydantic-ai==0.1.9
opentelemetry-sdk==1.29.0
opentelemetry-exporter-otlp-proto-http==1.29.0
opentelemetry-semantic-conventions==1.37.0

Workaround

Currently, the only way to see tool results is to:

  1. Comment out tracer_provider.add_span_processor(OpenInferenceSpanProcessor())
  2. View raw span attributes in Phoenix UI under the "Attributes" tab
  3. Look for tool_response (v2) or gen_ai.tool.call.result (v3) attributes

However, this removes the nice formatting and proper span categorization that OpenInferenceSpanProcessor provides.

luzmkk avatar Nov 17 '25 15:11 luzmkk

Hi @luzmkk! I'm Dosu and I’m helping the openinference team.

The root cause is that OpenInferenceSpanProcessor does not map tool result attributes like tool_response (v2) or gen_ai.tool.call.result (v3) from Pydantic AI spans to the OpenInference output fields expected by Phoenix UI. The processor merges only known OpenInference attributes and ignores these tool result fields, so they never appear as span.output or similar in Phoenix—even though they're present in the raw attributes. To fix this, the processor (or its semantic conventions) needs to explicitly detect these attributes and set them as output.value or the appropriate OpenInference output attribute so Phoenix can display them as tool results cite.

As a workaround, you could subclass or monkey-patch the processor to copy tool_response or gen_ai.tool.call.result into output.value on tool spans before merging attributes. This will make tool results visible in the Phoenix UI without losing OpenInference formatting and categorization. Recent PRs in OpenInference show ongoing improvements in tool output handling, so a future release may address this natively.

To reply, just mention @dosu.


How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other  Chat with Dosu Join Discord Share on X

dosubot[bot] avatar Nov 17 '25 15:11 dosubot[bot]