Unable to build from source for tag `v0.16.0`.

[jingzhaoou]

System Info

CPU: x86_64 GPU: L4 GPU memory: 24GB OS: Ubuntu

Who can help?

I checked out v0.15.0 and was able to build successfully using bash build.sh. However, I checked out v0.16.0 and built using bash build.sh and ran into the following error:

utils.cc:(.text+0x6758): undefined reference to `tensorrt_llm::executor::Request::Request(std::vector<int, std::allocator<int> >, int, bool, tensorrt_llm::executor::SamplingConfig const&, tensorrt_llm::executor::OutputConfig const&, std::optional<int> const&, std::optional<int> const&, std::optional<std::vector<int, std::allocator<int> > >, std::optional<std::__cxx11::list<std::vector<int, std::allocator<int> >, std::allocator<std::vector<int, std::allocator<int> > > > >, std::optional<std::__cxx11::list<std::vector<int, std::allocator<int> >, std::allocator<std::vector<int, std::allocator<int> > > > >, std::optional<tensorrt_llm::executor::Tensor>, std::optional<tensorrt_llm::executor::ExternalDraftTokensConfig>, std::optional<tensorrt_llm::executor::PromptTuningConfig>, std::optional<tensorrt_llm::executor::LoraConfig>, std::optional<tensorrt_llm::executor::LookaheadDecodingConfig>, std::optional<tensorrt_llm::executor::KvCacheRetentionConfig>, std::optional<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::optional<std::vector<int, std::allocator<int> > >, std::optional<unsigned long>, bool, float, tensorrt_llm::executor::RequestType, std::optional<tensorrt_llm::executor::ContextPhaseParams>, std::optional<tensorrt_llm::executor::Tensor>, std::optional<int>, std::optional<tensorrt_llm::executor::Tensor>, int, std::optional<tensorrt_llm::executor::EagleConfig>, std::optional<tensorrt_llm::executor::Tensor>)'
collect2: error: ld returned 1 exit status
gmake[2]: *** [CMakeFiles/triton-tensorrt-llm-backend.dir/build.make:160: libtriton_tensorrtllm.so] Error 1
gmake[2]: Leaving directory '/tmp/tritonbuild/tensorrtllm/build'
gmake[1]: Leaving directory '/tmp/tritonbuild/tensorrtllm/build'
gmake[1]: *** [CMakeFiles/Makefile2:271: CMakeFiles/triton-tensorrt-llm-backend.dir/all] Error 2
gmake: *** [Makefile:136: all] Error 2

Any help would be greatly appreciated.

In build.sh, I also needed to comment out the following line. Otherwise build.py would complain about invalid arguments.

https://github.com/triton-inference-server/tensorrtllm_backend/blob/7a56e091a788ccf042760cf2c63ea957efc398db/build.sh#L65

This seems to make sense as I did not define TRITON_GITHUB_ORGANIZATION, which ended up adding "" as the last argument of build.py. Note that this line is not in v0.15.0.

Any help would be appreciated.

@byshiue @schetlur-nv

Information

• [x] The official example scripts
• [ ] My own modified scripts

Tasks

• [x] An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• [ ] My own task or dataset (give details below)

Reproduction

See above. Just run bash build.py.

Expected behavior

The build to be successful without error.

actual behavior

The build failed during linking.

additional notes

N/A