llama-stack

llama-stack copied to clipboard

Run ollama gpu distribution failed

2

### System Info NVIDIA GPU A30 nvidia-smi ``` Thu Oct 31 11:43:51 2024 +-----------------------------------------------------------------------------------------+ | NVIDIA-SMI 555.42.02 Driver Version: 555.42.02 CUDA Version: 12.5 | |-----------------------------------------+------------------------+----------------------+ | GPU Name Persistence-M |...

alexhegit

Significantly simpler and malleable test setup

2

# What does this PR do? Significantly simplifies running tests. Previously you ran tests by doing: ```bash MODEL_ID= PROVIDER_ID= PROVIDER_CONFIG=config.yaml pytest -s llama_stack/providers/tests/inference/test_inference.py ``` This was pretty annoying because -...

ashwinb

# What does this PR do? - Update documentations ## Feature/Issue validation/testing/test plan - https://llama-stack.readthedocs.io/en/latest/ ## Sources Please link relevant resources if necessary. ## Before submitting - [ ] This...

yanxi0830

This is needed after the async refactor. testplan: * llama stack build --template bedrock --image-type conda * llama stack run bedrock --port 5000 * python examples/inference/client.py localhost 5000 ``` User>hello...

dineshyv

testing steps using conda: * llama stack build --template bedrock --image-type conda * llama stack configure bedrock * llama stack run bedrock --port 5000 * verified models are registered by...

dineshyv

LLamaGuard, routing, and vllm

6

### System Info Cuda 12.6, torch 2.5.1, nvidia gpu ### Information - [X] The official example scripts - [ ] My own modified scripts ### 🐛 Describe the bug I'm...

stevegrubb

This is a WIP to get quick feedback. Main things i need feedback on: 1) Accessing registry as a global variable. Other option is to pass along either the dist...

dineshyv

# What does this PR do? this PR adds a basic inference adapter to NVIDIA NIMs what it does not do - - support streaming - have test coverage for...

mattf

--continuation of https://github.com/meta-llama/llama-stack/pull/352 # TL;DR 1. Implement [OpenAI's SimpleQA's Benchmark](https://openai.com/index/introducing-simpleqa/) as ScoringFn ([reference](https://github.com/openai/simple-evals/blob/main/simpleqa_eval.py)) [RFC] - Option 1: SimpleQAScoringFn: Move each benchmark eval into separate scoring function with it's own context....

yanxi0830

--continuation of https://github.com/meta-llama/llama-stack/pull/333 # TL;DR 1. Introduce a registerable AnswerParsingScoringFn with AnswerParsingScoringContext for registering scoring functions with context 2. Remove parameters field (context alone is sufficient for things related to...

yanxi0830