[FEATURE] Add Anthropic LLM support via Anthropic Python SDK

[dot-agi]

📥 Pull Request

📘 Description This PR adds functions in the llm_tracker.py file for the asynchronous and synchronous Anthropic SDK support. An example notebook anthropic_example under examples/anthropic is also added to demo the integration.

🎯 Goal To extend the AgentOps capability to the Anthropic LLMs.

🔍 Additional Context Tool Calling in Anthropic Python SDK seems flaky and might not result in correct responses,

🧪 Testing Tested using the provided example notebook.

Thank you for your contribution to Agentops!

[gitguardian[bot]]

️✅ There are no secrets present in this pull request anymore.

If these secrets were true positive and are still valid, we highly recommend you to revoke them. Once a secret has been leaked into a git repository, you should consider it compromised, even if it was deleted immediately. Find here more information about risks.

---

^{_{🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.}}

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 0% with 668 lines in your changes missing coverage. Please review.

Files	Patch %	Lines
agentops/llm_tracker.py	0.00%	668 Missing :warning:

Flag	Coverage Δ
unittests	34.85% <0.00%> (-10.75%)	:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Files	Coverage Δ
agentops/event.py	92.00% <ø> (ø)
agentops/llm_tracker.py	0.00% <0.00%> (ø)

[dot-agi]

@bboynton97 @areibman This is ready for review!

[areibman]

Unfortunate timing for this PR. @bboynton97 is currently splitting llm_tracker.py into multiple files for readability. When that merge goes through, we'll make some changes to make this one work

[dot-agi]

Unfortunate timing for this PR. @bboynton97 is currently splitting llm_tracker.py into multiple files for readability. When that merge goes through, we'll make some changes to make this one work

I'd like to help, would be good for me to understand the working of the software in detail and I can use that information to draft a contributing guidelines document and/or observe where tests are needed.

[areibman]

Unfortunate timing for this PR. @bboynton97 is currently splitting llm_tracker.py into multiple files for readability. When that merge goes through, we'll make some changes to make this one work

I'd like to help, would be good for me to understand the working of the software in detail and I can use that information to draft a contributing guidelines document and/or observe where tests are needed.

This is the PR in question: https://github.com/AgentOps-AI/agentops/pull/346

From your end, this would just mean splitting the Anthropic handlers into its own file. The tricky part is just merging those changes from that branch into yours.

[areibman]

@HowieG Can you add the Anthropic notebook to the smoke test suite?

[dot-agi]

Unfortunate timing for this PR. @bboynton97 is currently splitting llm_tracker.py into multiple files for readability. When that merge goes through, we'll make some changes to make this one work

I'd like to help, would be good for me to understand the working of the software in detail and I can use that information to draft a contributing guidelines document and/or observe where tests are needed.

This is the PR in question: #346

From your end, this would just mean splitting the Anthropic handlers into its own file. The tricky part is just merging those changes from that branch into yours.

Cool, I will look into it and find a solution.

[HowieG]

@HowieG Can you add the Anthropic notebook to the smoke test suite?

Any notebook added to tests or examples folders will be smoke tested automatically. Except any we've excluded

[dot-agi]

Unfortunate timing for this PR. @bboynton97 is currently splitting llm_tracker.py into multiple files for readability. When that merge goes through, we'll make some changes to make this one work

I'd like to help, would be good for me to understand the working of the software in detail and I can use that information to draft a contributing guidelines document and/or observe where tests are needed.

This is the PR in question: #346 From your end, this would just mean splitting the Anthropic handlers into its own file. The tricky part is just merging those changes from that branch into yours.

Cool, I will look into it and find a solution.

I rebased and refactored the code. I believe that this PR should merge to llm-handler-refactor if that's where the changes are right now.

[bboynton97]

hey @the-praxs! thank you for reworking this, it looks fantastic! i'm going to go ahead and merge this into the llm-handler-refactor like you set it up to do and make sure you get credit on the subsequent PR as well :)

thank you for your work here! its very appreciated