mlc-llm
mlc-llm copied to clipboard
[Bug] Compiling the MLC from source is failed (cuda_fp8.h)
🐛 Bug
I execute the following scripts (from https://llm.mlc.ai/docs/install/mlc_llm.html#option-2-build-from-source) to build the MLC-llm from the source code, but it fails.
To Reproduce
# clone from GitHub
git clone --recursive https://github.com/mlc-ai/mlc-llm.git && cd mlc-llm/
# create build directory
mkdir -p build && cd build
# generate build configuration
python ../cmake/gen_cmake_config.py
# build mlc_llm libraries
cmake .. && cmake --build . --parallel $(nproc) && cd ..
Error:
./mlc-llm/3rdparty/tvm/3rdparty/flashinfer/src/generated/batch_paged_decode_head_128_logitshook_0_posenc_1_dtypeq_f16_dtypekv_f16_dtypeout_f16_idtype_i32.cu:1:
./mlc-llm/3rdparty/tvm/3rdparty/flashinfer/include/flashinfer/attention/../vec_dtypes.cuh:21:10: fatal error: cuda_fp8.h: No such file or directory
21 | #include <cuda_fp8.h>
| ^~~~~~~~~~~~
compilation terminated.
gmake[2]: *** [tvm/3rdparty/flashinfer/CMakeFiles/decode_kernels.dir/build.make:181: tvm/3rdparty/flashinfer/CMakeFiles/decode_kernels.dir/src/generated/batch_paged_decode_head_128_logitshook_0_posenc_1_dtypeq_f16_dtypekv_f16_dtypeout_f16_idtype_i32.cu.o] Error 1
gmake[2]: *** Waiting for unfinished jobs....
./mlc-llm/3rdparty/tvm/3rdparty/flashinfer/src/generated/single_decode_head_128_logitshook_1_posenc_1_dtypeq_f16_dtypekv_f16_dtypeout_f16.cu:1:
/home/wwt/mlc-llm/3rdparty/tvm/3rdparty/flashinfer/include/flashinfer/attention/../vec_dtypes.cuh:21:10: fatal error: cuda_fp8.h: No such file or directory
21 | #include <cuda_fp8.h>
| ^~~~~~~~~~~~
compilation terminated.
Expected behavior
I hope to receive help from kind-hearted people.
Environment
- Platform (e.g. CUDA):
- Operating system (e.g. Ubuntu):
- How you installed MLC-LLM (
conda, source): - Python version (3.11):
- GPU driver version (565.57.01):
- CUDA version (cuda_11.5.r11.5):