openinference icon indicating copy to clipboard operation
openinference copied to clipboard

[bug] blank output value in Agno traces

Open guyrosin opened this issue 1 month ago • 7 comments

Describe the bug In openinference-instrumentation-agno v0.1.22, traces are logged with a blank output value when using streaming.

Both the OUTPUT_VALUE and OUTPUT_MIME_TYPE attributes are blank. This seems like a regression from #2406. cc @dirkbrnd

To Reproduce Instrument Agno with OpenInference, send a message with streaming=True and check if the output field is populated.

Expected behavior The output field should contain the model's response.

Screenshots In the following screenshot, the second row used v0.1.22 and it's missing an output value. The first row was generated using v0.1.21. Image

Desktop (please complete the following information):

  • OS: macOS
  • Version: 26.0.1

Additional context I'm using Agno v2.2.13 and hosted Langfuse.

guyrosin avatar Nov 16 '25 12:11 guyrosin

This does look like a regression introduced in v0.1.22 as part of the changes for #2406, which refactored the Agno instrumentation internals by splitting the old _wrappers.py into multiple files and updating how runs are wrapped and parsed. The timing and symptoms match: v0.1.21 populates the output field, but v0.1.22 does not, and the only significant change in that release was the refactor for iterators and input parsing tied to #2406 (release notes, PR #2406).

It's likely that the new wrappers or the updated run handling logic are not correctly extracting or setting the output value on the span. This could be due to a missed assignment, a change in how outputs are accessed from Agno's run events, or a serialization issue.

As a next step, check if any environment variables like OPENINFERENCE_HIDE_OUTPUTS are set (they default to False, so outputs should be visible unless explicitly hidden). If not, the regression is almost certainly in the new wrapper logic. You may want to compare the output handling in v0.1.21's _wrappers.py with the new _model_wrapper.py, _runs_wrapper.py, and _tools_wrapper.py in v0.1.22 to spot where the output assignment diverged.

If you need a quick workaround, reverting to v0.1.21 should restore the expected output field until a fix is released.

To reply, just mention @dosu.


How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other  Chat with Dosu Join Discord Share on X

dosubot[bot] avatar Nov 16 '25 12:11 dosubot[bot]

Hi @guyrosin I have just tested with the latest openinference (0.1.23) (should be the same as 0.1.22) with the latest Agno (2.2.13) and could not replicate. My test looks something like this

asyncio.run(
    team.aprint_response(
        "What is the current price of Tesla? Then find the current price of NVIDIA",
            user_id=user_id,
            session_id=session_id,
            stream=True,
            stream_events=True,
        )
)

dirkbrnd avatar Nov 18 '25 08:11 dirkbrnd

Thanks @dirkbrnd. I just realized something that may help here: the ainvoke_stream event actually contains the model's output. Only the higher level arun event is missing the output...

Image Image

Here's my code:

mina_agent = Agent(
    name="Mina",
    model="openai:gpt-4o-mini",
)


await mina_agent.aprint_response(
    "this is streaming",
    user_id="[email protected]",
    stream=True,
)

guyrosin avatar Nov 20 '25 17:11 guyrosin

Hi @guyrosin this does look like an issue here. I am able to replicate and will work on this. I will fix it in one of my WIP PR here- https://github.com/Arize-ai/openinference/pull/2450

Thanks a lot for raising this!

kausmeows avatar Nov 21 '25 10:11 kausmeows

hi @guyrosin i have fixed this in my branch- https://github.com/Arize-ai/openinference/pull/2450

Image

kausmeows avatar Nov 21 '25 18:11 kausmeows

Root Cause Analysis

I've done some investigation and found the exact cause of this issue.

The Bug

In _runs_wrapper.py, the arun_stream method has this logic (line 506-508):

if "yield_run_output" not in kwargs:
    yield_run_output_set = True
    kwargs["yield_run_output"] = True

The problem is that Agno's public arun() method always passes yield_run_output explicitly to _arun_stream(), even when it's None:

# In agno/agent/agent.py line 2615-2621
return self._arun_stream(
    ...
    yield_run_output=yield_run_output,  # Always passed, even if None
    ...
)

So when a user calls agent.arun(stream=True):

  1. arun() receives yield_run_output=None (default)
  2. arun() calls _arun_stream(..., yield_run_output=None, ...)
  3. OpenInference sees "yield_run_output" in kwargsTrue (key exists, value is None)
  4. The condition "yield_run_output" not in kwargs is False, so it doesn't override
  5. No RunOutput is yielded, so run_response stays None

Proper Fix

The condition should check for falsy values, not just key presence:

# Current (buggy):
if "yield_run_output" not in kwargs:
    kwargs["yield_run_output"] = True

# Should be:
if not kwargs.get("yield_run_output"):
    kwargs["yield_run_output"] = True

About PR #2450

The final_response fallback in PR #2450 works, but it's a workaround rather than a fix:

  • It passes RunCompletedEvent to _extract_run_response_output() which expects RunOutput
  • Works via duck typing (both have .content), but RunOutput has richer metadata (metrics, run_id, messages)
  • The proper RunOutput object should be captured when available

Workaround

Until fixed, users can explicitly pass yield_run_output=True:

async for event in agent.arun(message, stream=True, yield_run_output=True):
    # process events...

This is a documented Agno API parameter and ensures the RunOutput is yielded for proper capture.

Reproducer

# Minimal reproducer showing the issue
from agno.agent import Agent
from agno.models.openai import OpenAIChat
from agno.run.agent import RunOutput

agent = Agent(model=OpenAIChat(id="gpt-4o-mini"))

# BUG: No RunOutput in stream
async for event in agent.arun("Hello", stream=True):
    print(type(event).__name__, isinstance(event, RunOutput))
# Output: RunStartedEvent False, RunContentEvent False, ..., RunCompletedEvent False

# WORKAROUND: RunOutput is yielded
async for event in agent.arun("Hello", stream=True, yield_run_output=True):
    print(type(event).__name__, isinstance(event, RunOutput))
# Output: ..., RunCompletedEvent False, RunOutput True

durandom avatar Nov 22 '25 09:11 durandom

Thanks for the dive in @durandom that does make sense, i have done updates to my PR!

kausmeows avatar Nov 22 '25 09:11 kausmeows

When can we expect this to be fixed

hudyweas avatar Dec 02 '25 10:12 hudyweas

When can we expect this to be fixed

Hey @hudyweas its fixed here- https://github.com/Arize-ai/openinference/pull/2450, just waiting on the Arize team to release it Expect it out by this week at max

kausmeows avatar Dec 02 '25 10:12 kausmeows