Trying to compile the latest trtllm (under the v0.12 main branch) in triton 24.07-trtllm-python-py3 reports an error

[gzy19990617]

System Info

triton images:24.07-trtllm-python-py3

Who can help?

No response

Information

• [ ] The official example scripts
• [ ] My own modified scripts

Tasks

• [ ] An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• [ ] My own task or dataset (give details below)

Reproduction

The image comes with v0.11 trtllm, but v0.11 does not support QWEN2-57b, so I want to compile trtllm to the latest main branch, but it cannot be compiled successfully.

Expected behavior

get trtllm whl

actual behavior

[ 80%] Built target context_attention_src [ 98%] Built target decoder_attention_src [ 98%] Built target check_symbol_executor [ 98%] Built target check_symbol [ 98%] Built target tensorrt_llm [100%] Built target nvinfer_plugin_tensorrt_llm [100%] Built target executorWorker Requirement already satisfied: pybind11-stubgen in /usr/local/lib/python3.10/dist-packages (2.5.1) WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable.It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning. Traceback (most recent call last): File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main return _run_code(code, main_globals, None, File "/usr/lib/python3.10/runpy.py", line 86, in _run_code exec(code, run_globals) File "/usr/local/lib/python3.10/dist-packages/pybind11_stubgen/main.py", line 4, in main() File "/usr/local/lib/python3.10/dist-packages/pybind11_stubgen/init.py", line 319, in main run( File "/usr/local/lib/python3.10/dist-packages/pybind11_stubgen/init.py", line 358, in run QualifiedName.from_str(module_name), importlib.import_module(module_name) File "/usr/lib/python3.10/importlib/init.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "", line 1050, in _gcd_import File "", line 1027, in _find_and_load File "", line 1006, in _find_and_load_unlocked File "", line 674, in _load_unlocked File "", line 571, in module_from_spec File "", line 1176, in create_module File "", line 241, in _call_with_frames_removed ImportError: /gaoziyuan/gzy/tensorrtllm_backend/tensorrt_llm/tensorrt_llm/bindings.cpython-310-x86_64-linux-gnu.so: undefined symbol: _ZN12tensorrt_llm8executor14ExecutorConfig25setLogitsPostProcessorMapERKSt13unordered_mapINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt8functionIFvmRNS0_6TensorERKSt6vectorISC_IiSaIiEESaISE_EERKSt10shared_ptrINS_7runtime10CudaStreamEESt8optionalImEEESt4hashIS8_ESt8equal_toIS8_ESaISt4pairIKS8_SS_EEE Failed to build pybind11 stubgen: Command '"/usr/bin/python3" -m pybind11_stubgen -o . bindings' returned non-zero exit status 1.

• Building wheel... Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/pyproject_hooks/_in_process/_in_process.py", line 373, in main() File "/usr/local/lib/python3.10/dist-packages/pyproject_hooks/_in_process/_in_process.py", line 357, in main json_out["return_val"] = hook(**hook_input["kwargs"]) File "/usr/local/lib/python3.10/dist-packages/pyproject_hooks/_in_process/_in_process.py", line 271, in build_wheel return _build_backend().build_wheel( File "/usr/local/lib/python3.10/dist-packages/setuptools/build_meta.py", line 415, in build_wheel return self._build_with_temp_dir( File "/usr/local/lib/python3.10/dist-packages/setuptools/build_meta.py", line 397, in _build_with_temp_dir self.run_setup() File "/usr/local/lib/python3.10/dist-packages/setuptools/build_meta.py", line 497, in run_setup super().run_setup(setup_script=setup_script) File "/usr/local/lib/python3.10/dist-packages/setuptools/build_meta.py", line 313, in run_setup exec(code, locals()) File "", line 84, in File "", line 48, in sanity_check ImportError: The bindings module does not exist. Please check the package integrity. If you are attempting to use the pip development mode (editable installation), please execute build_wheels.py first, and then run pip install -e ..

ERROR Backend subprocess exited when trying to invoke build_wheel Traceback (most recent call last): File "/gaoziyuan/gzy/tensorrtllm_backend/tensorrt_llm/./scripts/build_wheel.py", line 391, in main(**vars(args)) File "/gaoziyuan/gzy/tensorrtllm_backend/tensorrt_llm/./scripts/build_wheel.py", line 304, in main build_run( File "/usr/lib/python3.10/subprocess.py", line 526, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '"/usr/bin/python3" -m build /gaoziyuan/gzy/tensorrtllm_backend/tensorrt_llm --skip-dependency-check --no-isolation --wheel --outdir "/gaoziyuan/gzy/tensorrtllm_backend/tensorrt_llm/build"' returned non-zero exit status 1.

additional notes

who can help me.thanks!