llama-stack

llama-stack copied to clipboard

--continuation of https://github.com/meta-llama/llama-stack/pull/323 # TL;DR - Add LLM as judge meta-reference impl. Judge using Llama Stack inference_api. - Move ScoringFnDef to `*.json` based files for easier registration. - Support dynamic...

yanxi0830

Guardrail Loading Failed with Unexpected Large GPU Memory Requirement at Multi-GPU Server

7

### System Info Python version: 3.10.12 Pytorch version: llama_models version: 0.0.42 llama_stack version: 0.0.42 llama_stack_client version: 0.0.41 Hardware: 4xA100 (40GB VRAM/GPU) local-gpu-run.yaml file content is as following: ``` version: '2'...

dawenxi-007

What configs input when build from distributions/meta-reference-gpu/build.yaml

4

### System Info NVIDIA A30 ### Information - [ ] The official example scripts - [ ] My own modified scripts ### 🐛 Describe the bug I try build by...

AlexHe99

### System Info - PyTorch version: 2.2.2 - Is debug build: False - CUDA used to build PyTorch: None - ROCM used to build PyTorch: N/A - - OS: macOS...

cheesecake100201

### 🚀 The feature, motivation and pitch Add a distribution sample to support AMD ROCm GPU like `meta-reference-gpu` which looks like just support Nvidia GPU ### Alternatives _No response_ ###...

AlexHe99

docker image name and GPU issue

3

I am using the latest version I just pip installed 0.0.45. In my environment (Fedora 39) I have export DOCKER_BINARY="podman" When I build, pretty much following the example, I get...

stevegrubb

I can work on this

cheesecake100201

How to specify the model type using the pre-build docker?

15

### System Info Using Windows 11 ### Information - [X] The official example scripts - [X] My own modified scripts ### 🐛 Describe the bug When running: ``` docker run...

Travis-Barton

Ollama inference issue - llama3.2 not registered

10

### System Info Ubuntu, CPU only, Conda, Python 3.10 ### Information - [x] The official example scripts - [ ] My own modified scripts ### 🐛 Describe the bug I...

akhtet

### System Info Linux Ubuntu Anaconda ### Information - [X] The official example scripts - [ ] My own modified scripts ### 🐛 Describe the bug llama stack build /tmp/a/llama/anaconda/envs/stack/lib/python3.10/site-packages/pydantic/_internal/_fields.py:172:...

JoseGuilherme1904