agenta
agenta copied to clipboard
Agenta-AI
[Enhancement]: add integration with litellm for observability
Description
This PR allows litellm integration for observability.
Related Issue
Closes #1582
Additional Information
The example demonstrating LiteLLM integration for observability is titled "litellm_integration". This example is included in the pull request.
The latest updates on your projects. Learn more about Vercel for Git ↗︎
| Name | Status | Preview | Comments | Updated (UTC) |
|---|---|---|---|---|
| agenta | ✅ Ready (Inspect) | Visit Preview | 💬 Add feedback | Jun 6, 2024 2:13pm |
![vercel[bot] avatar](https://avatars.githubusercontent.com/in/8329?v=4 "Published by a pub.dev verified publisher"){kind=small}
The input for this span is totally wrong. Let's look at how others process the data from litellm and copy it ( a note here, everyone seems to be using the response_obj while we are using kwargs, why?)
They're not. Check out this and this. The kwargs in log_pre_api_call contain the prepared data for the LLM provider. We only have access to response_obj after receiving a response from the LLM provider.
re: @mmabrouk
The input for this span is totally wrong. Let's look at how others process the data from litellm and copy it ( a note here, everyone seems to be using the response_obj while we are using kwargs, why?)
They're not. Check out this and this. The kwargs in
log_pre_api_callcontain the prepared data for the LLM provider. We only have access toresponse_objafter receiving a response from the LLM provider.re: @mmabrouk
You are right. Sorry for missing that. However, the input for the span in the screenshot were wrong. It could be that the format that we are logging then is wrong (related to the issue we discussed this morning).
The input for this span is totally wrong. Let's look at how others process the data from litellm and copy it ( a note here, everyone seems to be using the response_obj while we are using kwargs, why?)
They're not. Check out this and this. The kwargs in
log_pre_api_callcontain the prepared data for the LLM provider. We only have access toresponse_objafter receiving a response from the LLM provider. re: @mmabroukYou are right. Sorry for missing that. However, the input for the span in the screenshot were wrong. It could be that the format that we are logging then is wrong (related to the issue we discussed this morning).
Yup, correct. This has been resolved.