always-H
always-H
hi, how you run dp? on single-node or multi-node?
@aubreyli 请问是不是因为gguf的bf16版本和硬件中的bf16版本数据格式布局不同？否则直接使用huggingface上safetensor的bf16版本不行吗
Seems related to [https://github.com/vllm-project/vllm/issues/6145#issuecomment-2211562438](url), but is it the same when using multi-node?
@njhill thx for commenting. I still have one question that is EP is usually used along with DP, but the only way to use DP+EP is through 'data_parallel.py' (as far...