Xiaosong Wang

icon company Beijing Language and Culture University icon location student of Beijing Language and Culture University

Results 1 issues of Xiaosong Wang

VLLM部署api自动使用Ray集群，部署失败

1 comment

### Reminder - [X] I have read the README and searched the existing issues. ### Reproduction CUDA_VISIBLE_DEVICES=0,1 API_PORT=8003 llamafactory-cli api --model_name_or_path /data1/llama_model/Meta-Llama-3-8B-Instruct --template llama3 --infer_backend vllm --vllm_enforce_eager true 这个命令我在本地的时候没有使用 Ray...