logicbaby
Results
1
comments of
logicbaby
On m1 macos 16G + vllm-0.8.2 success Qwen2.5-1.5B-Instruct add parameter `--dtype float32`, complete startup command: ``` HF_ENDPOINT=https://hf-mirror.com VLLM_USE_V1=0 vllm serve Qwen/Qwen2.5-1.5B-Instruct --dtype float32 ``` DeepSeek-R1-Distill-Qwen-1.5B need parameter `--dtype float32 --max-model-len...