open-webui
Add support for Arize Phoenix for LLM observability and evaluation
I am a regular user of Open WebUI and find it incredibly valuable for our team's LLM workflows. I noticed that while there's current support for Langfuse for tracing and evaluation, it would be great if there were an integration with Arize Phoenix as well. Arize Phoenix is also an open-source observability tool for LLM tracing and evaluation that uses Open Telemetry, which is used in Open WebUI's backend. We are currently using Phoenix in our stack alongside Open WebUI, and having direct integration would streamline our workflow significantly.
Looking at how the Langfuse integration is implemented, adding support for Arize Phoenix should be relatively straightforward, as Phoenix offers a Python SDK that makes it easier to log traces, evaluate model outputs, and visualize performance metrics.
I would be glad to provide more information or even contribute with a PR myself if needed.
Any updates on this? SInce Langfuse consumes a high amount of CPU (due to Clickhouse), it'd be great if some other tools like Phoenix (by Arize) could be integrated as well.
I am experimenting with Pheonix as well. I see several ways you could integrate phoenix directly.
- Via an LLM proxy like litellm (supported by phoenix)
- Via filter as you noted the langfuse example does
- Phoenix Auto instrumentation for all your pipelines.
The latter is quite simple to add for e.g. Azure OpenAI calls if you have a utils function that establishes a client for all your pipelines. (I am not a maintainer and this is probably not the integration you were hoping for, but it might help you)
Any news on integrating Phoenix using a filter? I am experiencing problems with implementing my own pipeline to do so.
