kuangdao

Results 15 comments of kuangdao

[BUG]nvcc fatal when DS_BUILD_TRANSFORMER_INFERENCE=1

0.9.1 error bug 0.9.0 works

how to train llama2-7b in a100 80G gpu?

dose not support zero of jax？

FSDP reduce_scatter can not overlap with compute

no one concerned about this issue？

tp_overlap need tensor parallel is equal world size ?

i have set export NCCL_DEBUG=WARN and there is no additional message

tp_overlap need tensor parallel is equal world size ?

thanks, i know, i think comm+GEMM overlap is outstanding job, and i hope more documents such as design and Implementation will be give.

[Bug] DeepSeek-R1 NCCL WatchDog Timeout Error

m

[Bug]: CUDA illegal memory access error when `enable_prefix_caching=True`

I had the same issue in deepseek, i use enable_prefix_caching=True , fp8 and tp = 2 vllm version is 0.5.1

[Bug]: CUDA illegal memory access error when `enable_prefix_caching=True`

0.5.3 still meet error

[Bug] deepseekr1 illegal memory access on 2*8*H20

mark

[Bug] DeepSeek-V3 an illegal memory access was encountered

mark