dd-trace-py icon indicating copy to clipboard operation
dd-trace-py copied to clipboard

[BUG]: LLMobs decorator causes exception if decorating a function that returns a pandas dataframe

Open kyledemeule opened this issue 6 months ago • 0 comments

Tracer Version(s)

3.7.0

Python Version(s)

Python 3.13.1

Pip Version(s)

pip 25.0.1

Bug Report

We're using DataDog LLM Observability to observe our Langgraph agent. We use the ddtrace.llmobs.decorators.tool decorator for our tool call. If the tool call returns a dataframe, we get the following exception:

  File "/copilot/.venv/lib/python3.13/site-packages/ddtrace/llmobs/decorators.py", line 249, in wrapper
    and resp
        ^^^^
  File "/copilot/.venv/lib/python3.13/site-packages/pandas/core/generic.py", line 1577, in __nonzero__
    raise ValueError(
    ...<2 lines>...
    )
ValueError: The truth value of a DataFrame is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all().

And the tool call of our agent fails.

ddtrace/llmobs/decorators.py puts the function response in an if statement, but pandas dataframes can't be coerced to booleans like that:

>>> import pandas as pd
>>> if(pd.DataFrame()):
...     print(1)
...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/kyledemeule/.virtualenvs/pipe2/lib/python3.12/site-packages/pandas/core/generic.py", line 1577, in __nonzero__
    raise ValueError(
ValueError: The truth value of a DataFrame is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all().

As an aside, do we expect errors from llmobs decorators to cause our code to fail? This bug affects the ability of our agent to respond to our users input. Is it possible to silence exceptions from LLM Observability? If the tracing has errors we would prefer the tracing doesn't run instead of causing our code to error.

Reproduction Code

import pandas as pd
from ddtrace.llmobs.decorators import tool

@tool
def tool_test():
  return pd.DataFrame()

tool_test()

Error Logs

  File "/copilot/.venv/lib/python3.13/site-packages/ddtrace/llmobs/decorators.py", line 249, in wrapper
    and resp
        ^^^^
  File "/copilot/.venv/lib/python3.13/site-packages/pandas/core/generic.py", line 1577, in __nonzero__
    raise ValueError(
    ...<2 lines>...
    )
ValueError: The truth value of a DataFrame is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all().

Libraries in Use

No response

Operating System

No response

kyledemeule avatar Jun 09 '25 18:06 kyledemeule