skypilot icon indicating copy to clipboard operation
skypilot copied to clipboard

Running Docker on RunPod doesn't work

Open okdewit opened this issue 4 months ago • 2 comments

I used Skypilot docs and Mistral docs to create this YAML:

resources:
  accelerators: RTXA4000:1

run: |
  docker run --gpus all -p 8000:8000 ghcr.io/mistralai/mistral-src/vllm:latest \
                   --host 0.0.0.0 \
                   --model mistralai/Mistral-7B-Instruct-v0.2 \
                   --tensor-parallel-size 1

This spins up runpod/base:0.0.2 docker image, and I think it then tries run docker within that container, which won't work.

Is there a way to deploy docker images on RunPod through Skypilot, or maybe create a RunPod template through SkyPilot?

okdewit avatar Feb 06 '24 09:02 okdewit