llama-stack icon indicating copy to clipboard operation
llama-stack copied to clipboard

add nvidia distribution

Open cdgamarose-nv opened this issue 11 months ago • 1 comments
trafficstars

What does this PR do?

adds nvidia template for creating a distribution using inference adapter for NVIDIA NIMs.

Test Plan

Please describe: Build llama stack distribution for nvidia using the template, docker and conda.

llama stack build --template nvidia

or

cd distributions/inline-nvidia/ && docker compose up

Before submitting

  • [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • [ ] Ran pre-commit to handle lint / formatting issues.
  • [x] Read the contributor guideline, Pull Request section?
  • [x] Updated relevant documentation.
  • [ ] Wrote necessary unit or integration tests.

cdgamarose-nv avatar Dec 04 '24 03:12 cdgamarose-nv