Vector Zhou

icon indicating a website link vectorzhou.com

icon company Paul G. Allen School of Computer Science & Engineering, University of Washington icon location Seattle icon location 越下青尘越不休

Results 1 comments of Vector Zhou

[Bug] RuntimeError: RMSNorm failed with error code invalid configuration argument

Same issue when hosting `DeepSeek-R1-Distill-Qwen-14B` with commands `python3 -m sglang.launch_server --model-path ${MODEL_PATH} --tp 8 --dist-init-addr ${IP}:5000 --trust-remote-code --host 0.0.0.0 --port 30000 --enable-dp-attention --dp-size 8 --enable-torch-compile --torch-compile-max-bs 8` on 8 H100s.