Kevin Tang

icon indicating a website link https://www.kevin125.com icon indicating an email [email protected]

icon company Intel icon location Shanghai

Results 2 comments of Kevin Tang

vllm can‘t use oneCCL on host

oneAPI: l_BaseKit_p_2024.0.1.46_offline.sh conda env: ipex-vllm https://github.com/intel-analytics/ipex-llm/blob/66fe2ee46465306e241296b2d3440f6ba31b7305/docs/mddocs/Quickstart/vLLM_quickstart.md

fix rocm build error

Looks Good, and one suggestion for runtime exception: undefined symbol: _Z16gptq_marlin_gemmRN2at6TensorES1_S1_S1_S1_S1_lS0_lllib ``` --- a/csrc/custom_marlin/gptq_marlin/gptq_marlin.cu +++ b/csrc/custom_marlin/gptq_marlin/gptq_marlin.cu @@ -74,8 +74,8 @@ namespace gptq_marlin { torch::Tensor gptq_marlin_gemm(torch::Tensor& a, torch::Tensor& b_q_weight, torch::Tensor& b_scales,...