burr
burr copied to clipboard
apache
Prompt playground
Allows to load traces from tracked Burr application with OpenLLMetry traces.
You can load previous chat interactions and try new prompts with multiple LLM providers via LiteLLM.
To launch the app
OPENAI_API_KEY=sk-... ANTHROPIC_API_KEY=sk-ant-... streamlit run burr/integrations/playground/app.py
Features:
- 3 providers at once
- load interactions directly from local Burr tracking
- load a specific interaction via the URL params
- the playground interactions themselves are logged using Burr under
burr-playground
Limitations:
- streamlit makes it very inconvenient to call LLMs asynchronously
- currently no streamlined way to support generic API key input
OK, looking good. A few minor UI points after playing around:
- Model not selected doesn't run it -- maybe we should be able to schose an arbitrary number of them? Or just grey it out if it's not selected?
- Launch button is ina. weird place, I think it should be below the provider/model selection
- Selecting a model resets everything -- it shouldn't reset the other ones -- we shoud have a reset button
- We should have a progress button if it's running
- The third one never seems to work for me...
- Tab to link to the burr UI (iframe?) -- we should be able to see the trace it came from + the trace it led to See what I'm working with:
