Is there a docker container I can use?

[sungkim11]

I am looking for something like this, so I can run this on a single 3090.

docker run --gpus 1
-e HF_TOKEN=$HF_TOKEN -p 8000:8000
ghcr.io/mistralai/mistral-src/vllm:latest
--host 0.0.0.0
--model mistralai/Mistral-7B-Instruct-v0.2

[imoneoi]

Here is a beta version wrapping the official openchat server in this repo. We're working on a stable release and corresponding instructions.

https://hub.docker.com/r/ochat/openchat-server

[sungkim11]

I cannot seem to run this. It keeps on asking for model where I supplied the model's name both parameter and environment variable. Can you provide an instruction to run this docker image?