agentops icon indicating copy to clipboard operation
agentops copied to clipboard

OpenAI Agents SDK Guardrail Span Classification Issues

Open devin-ai-integration[bot] opened this issue 6 months ago • 0 comments

OpenAI Agents SDK Guardrail Span Classification Issues

Problem Description

The OpenAI Agents SDK integration has issues with guardrail span classification and duplicate span creation:

  1. Incorrect Span Classification: Guardrail events were being incorrectly classified as generic events or agent spans instead of proper GUARDRAIL span types
  2. Duplicate Span Creation: Each guardrail creates 2 spans: {guardrail_name} and {guardrail_name}.guardrail, making it unclear which span represents the actual guardrail execution
  3. Lost Usage Metrics: Recent fixes removed the wrong span, causing usage metrics to be lost from guardrail spans

Root Cause

Two functions in the OpenAI Agents SDK instrumentation were missing handlers for GuardrailSpanData:

  1. get_span_kind() in exporter.py - didn't handle GuardrailSpanData
  2. get_span_attributes() in attributes/common.py - didn't have a case for GuardrailSpanData

Expected Behavior

  • Guardrail events should be properly classified as GUARDRAIL spans
  • Each guardrail should create only one span with proper usage metrics
  • Guardrail spans should have appropriate attributes and observability data

Current Status

PR #1096 attempted to fix this but introduced new issues with duplicate spans and lost usage metrics.

Acceptance Criteria

  • [ ] Guardrail events are properly classified as GUARDRAIL span type
  • [ ] No duplicate spans are created for guardrails
  • [ ] Usage metrics are preserved in guardrail spans
  • [ ] Proper attributes are set for guardrail spans
  • [ ] Integration tests verify correct behavior

Related

  • PR #1096: Fix OpenAI Agents SDK guardrail span classification