agentops icon indicating copy to clipboard operation
agentops copied to clipboard

Missing @record_function decorator and AutoGen instrumentation reliability issues in v0.4.14

Open devin-ai-integration[bot] opened this issue 6 months ago • 0 comments

Missing @record_function decorator and AutoGen instrumentation reliability issues in v0.4.14

Problem Description

We've identified two critical issues affecting users upgrading to AgentOps v0.4.14:

  1. Missing @record_function decorator: The @record_function decorator that was available in previous versions is completely missing from v0.4.14, breaking existing user code that relies on manual instrumentation.

  2. AutoGen instrumentation reliability: AutoGen support isn't working as expected due to incorrect version requirements and package naming confusion in the instrumentation configuration.

Issue 1: Missing @record_function decorator

Expected Behavior

Users should be able to use the @record_function decorator to manually instrument functions, as was available in previous versions.

Current Behavior

The @record_function decorator is completely missing from the codebase. Attempting to import it results in:

ImportError: cannot import name 'record_function' from 'agentops'

Impact

  • Breaks existing user code that relies on manual instrumentation
  • No migration path or backward compatibility provided
  • No documentation on what replaced this functionality

Issue 2: AutoGen instrumentation reliability

Expected Behavior

AutoGen instrumentation should work reliably with current AutoGen package versions and automatically track agent interactions.

Current Behavior

  • Minimum version requirement is set to "0.1.0" which is too low for real AutoGen packages
  • Package naming confusion between "autogen", "ag2", and "autogen_agentchat" packages
  • Import errors in examples indicate version compatibility issues
  • Instrumentation may not activate due to incorrect version checks

Evidence

  1. Incorrect version requirement:

    # In agentops/instrumentation/__init__.py
    "autogen": {"module_name": "agentops.instrumentation.ag2", "class_name": "AG2Instrumentor", "min_version": "0.1.0"}
    

    The minimum version "0.1.0" is too low - AutoGen packages typically start from 0.3.x

  2. Import errors in examples:

    # In examples/ag2/agentchat_with_memory.py
    from autogen import ConversableAgent  # This import fails
    
  3. Package ecosystem confusion: The instrumentation targets "autogen" package but the ecosystem has evolved to include "ag2" and "autogen_agentchat" packages

Root Cause Analysis

@record_function Issue

  • The decorator was removed during the v4.x migration without providing backward compatibility
  • The functionality was replaced by the @tool decorator but no migration guide was provided
  • No deprecation warning was added to help users transition

AutoGen Issue

  • The AutoGen ecosystem has evolved with multiple package variants (autogen, ag2, autogen_agentchat)
  • Version requirements haven't been updated to reflect real package versions
  • Examples use inconsistent import patterns indicating version compatibility problems

Proposed Solution

1. Restore @record_function with backward compatibility

  • Add @record_function decorator to the legacy module with deprecation warning
  • Make it wrap the new @tool decorator for functionality
  • Export it in the main __init__.py for backward compatibility

2. Fix AutoGen instrumentation

  • Update minimum version requirements to realistic values (0.3.2 for autogen)
  • Add support for multiple AutoGen package variants (autogen, ag2)
  • Fix import issues in examples

User Impact

  • High: Existing code using @record_function is completely broken
  • Medium: AutoGen users may experience unreliable instrumentation
  • Migration: Users need clear guidance on transitioning from @record_function to @tool

Testing Strategy

  • Verify @record_function works with deprecation warning
  • Test AutoGen instrumentation with corrected version requirements
  • Run AutoGen examples to ensure they work correctly
  • Verify session tracking generates proper AgentOps links

This issue affects users who rely on manual instrumentation and AutoGen integration, making it a blocking issue for upgrading to v0.4.14.