openllmetry icon indicating copy to clipboard operation
openllmetry copied to clipboard

feat(instrumentation): Add event-based tracking implementation across providers

Open onyedikachi-david opened this issue 10 months ago • 10 comments

/claim #2456 Fixes: #2456

  • [x] I have added tests that cover my changes.
  • [ ] If adding a new instrumentation or changing an existing one, I've added screenshots from some observability platform showing the change.
  • [x] PR name follows conventional commits format: feat(instrumentation): ... or fix(instrumentation): ....
  • [ ] (If applicable) I have updated the documentation accordingly.

Description

This PR adds event-based tracking implementation across multiple providers while maintaining backward compatibility with the legacy attribute-based approach.

Completed Implementations:

  • ✅ Anthropic
  • ✅ Bedrock
  • ✅ Cohere
  • ✅ Google GenerativeAI
  • ✅ Groq
  • ✅ Langchain
  • ✅ LlamaIndex
  • ✅ MistralAI
  • ✅ Ollama
  • ✅ Replicate

Pending Implementations:

  • 🔄 SageMaker
  • 🔄 Together
  • 🔄 Transformers
  • 🔄 VertexAI
  • 🔄 WatsonX

Key Changes:

  1. Added event-based configuration with use_legacy_attributes support
  2. Implemented event utilities for:
    • Prompts
    • Completions
    • Embeddings (where applicable)
    • Tool/Function calls (where applicable)
  3. Added comprehensive test coverage for event emission
  4. Added support for both sync and async clients
  5. Added streaming support where applicable
  6. Maintained backward compatibility with legacy mode

Each provider implementation includes:

  • Event utilities (events.py)
  • Configuration updates (config.py)
  • Core instrumentation updates (__init__.py)
  • Comprehensive test coverage (test_events.py)

onyedikachi-david avatar Jan 22 '25 14:01 onyedikachi-david

CLA assistant check
All committers have signed the CLA.

CLAassistant avatar Jan 22 '25 14:01 CLAassistant

@nirga Could i get a review, to confirm I am on the right track. Thanks.

onyedikachi-david avatar Jan 22 '25 15:01 onyedikachi-david

@onyedikachi-david can you rebase so we can run the CI?

nirga avatar Jan 22 '25 15:01 nirga

@onyedikachi-david can you rebase so we can run the CI?

alright

onyedikachi-david avatar Jan 22 '25 16:01 onyedikachi-david

Done @nirga

onyedikachi-david avatar Jan 22 '25 17:01 onyedikachi-david

@onyedikachi-david looks like your tests are failing - can you take a look?

nirga avatar Feb 03 '25 20:02 nirga

@onyedikachi-david looks like your tests are failing - can you take a look?

Okay

onyedikachi-david avatar Feb 04 '25 01:02 onyedikachi-david

Hi, @nirga

Hope you don't mind I open another PR, the conflicts are a lot. I think it will be more cleaner. Thanks. What do you think?

onyedikachi-david avatar Feb 15 '25 21:02 onyedikachi-david

Sure @onyedikachi-david

nirga avatar Feb 15 '25 21:02 nirga

Thank you. On it.

onyedikachi-david avatar Feb 15 '25 21:02 onyedikachi-david