llama-stack

llama-stack copied to clipboard

feat: Add experimental integration tests with cached providers

1

# What does this PR do? Introduce a new workflow to run integration and agent tests against the OpenAI provider, and a subset of inference tests against Fireworks AI. Responses...

derekhiggins

### 🚀 Describe the new functionality needed I would like to use OpenTelemetry-Tracing to track my agent's activities. I created an agent as follows and registered a local 'calculator' and...

frzifus

Add a new lifecycle method to providers after the stack has been initialized

2

### 🚀 Describe the new functionality needed As I was writing an inference provider that needs to dynamically register models while starting (not through static declaration in run.yaml), I got...

jeffmaury

Llama Stack Playground

2

### 🚀 Describe the new functionality needed - Spec and build what is needed to build the v1 version of the playground ### 💡 Why is this needed? What if...

hardikjshah

Endpoint testing for hosted endpoints

2

### 🚀 Describe the new functionality needed All endpoints need a way to be tested and sent back to the endpoint to see what is not working. (25 examples) ###...

hardikjshah

Agent workflow dependency graph

1

### 🚀 Describe the new functionality needed We want to be able to keep track of message passing flow in multi-agent workflows. e.g. https://github.com/meta-llama/llama-stack/blob/main/docs/notebooks/Llama_Stack_Agent_Workflows.ipynb 1/ expose api to keep track...

yanxi0830

Improve CRUD on scoring function llm-as-judge

2

### 🚀 Describe the new functionality needed We need to refactor llm-as-judge to make it easy for user to perform CRUD operations. - unregister scoring functions - persisting judge prompts...

yanxi0830

Agent doesn't always select custom tool, even with `tool_choice="required"`

5

### System Info :128: RuntimeWarning: 'torch.utils.collect_env' found in sys.modules after import of package 'torch.utils', but prior to execution of 'torch.utils.collect_env'; this may result in unpredictable behaviour Collecting environment information... PyTorch...

anujsingla

prompt_guard blocks the server while running model

1

### System Info Collecting environment information... PyTorch version: 2.6.0+cu124 Is debug build: False CUDA used to build PyTorch: 12.4 ROCM used to build PyTorch: N/A OS: Debian GNU/Linux 12 (bookworm)...

derekhiggins

add token usage metadata to agents turn execution results

2

### 🚀 Describe the new functionality needed add token usage metadata from inference steps to agents when getting turn execution results ### 💡 Why is this needed? What if we...

jhpiedrahitao