xFasterTransformer icon indicating copy to clipboard operation
xFasterTransformer copied to clipboard
verified publisher icon intel

undefined reference to Qwen2LLM

Open Jasonjunsu opened this issue 7 months ago • 6 comments

when I build the source code on ubuntu 22.04, I have a issue as below, how do I fix it ? BTW, I used main branch. HEAD is e73e4c1ac03f44fe986f34c01bb345e8bc5409b4

/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), decoder_Qwen3MOELLM_nf4x2_t_float16_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x1a291): undefined reference to `Qwen2LLM<nf4x2_t, float16_t, Qwen3MOE<nf4x2_t, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), decoder_Qwen3MOELLM_int8_t_int8_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x1a351): undefined reference to `Qwen2LLM<signed char, signed char, Qwen3MOE<signed char, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), decoder_Qwen3MOELLM_bfloat16_t_int8_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x1a651): undefined reference to `Qwen2LLM<bfloat16_t, signed char, Qwen3MOE<bfloat16_t, bfloat16_t, bfloat16_t, bfloat16_t> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), decoder_Qwen3MOELLM_bfloat16_t_float16_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x1a7d1): undefined reference to `Qwen2LLM<bfloat16_t, float16_t, Qwen3MOE<bfloat16_t, bfloat16_t, bfloat16_t, bfloat16_t> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), decoder_Qwen3MOELLM_uint4x2_t_float16_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x1a951): undefined reference to `Qwen2LLM<uint4x2_t, float16_t, Qwen3MOE<uint4x2_t, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), decoder_Qwen3MOELLM_float16_t_int8_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x1aa11): undefined reference to `Qwen2LLM<float16_t, signed char, Qwen3MOE<float16_t, float16_t, float16_t, float16_t> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), decoder_Qwen3MOELLM_float16_t_float16_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x1ab91): undefined reference to `Qwen2LLM<float16_t, float16_t, Qwen3MOE<float16_t, float16_t, float16_t, float16_t> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), decoder_Qwen3MOELLM_uint4x2_t_int8_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x1add1): undefined reference to `Qwen2LLM<uint4x2_t, signed char, Qwen3MOE<uint4x2_t, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), decoder_Qwen3MOELLM_w8a8_t_int8_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x1bfd1): undefined reference to `Qwen2LLM<w8a8, signed char, Qwen3MOE<w8a8, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), decoder_Qwen3MOELLM_w8a8_t_float16_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x1c151): undefined reference to `Qwen2LLM<w8a8, float16_t, Qwen3MOE<w8a8, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), decoder_Qwen3MOELLM_int8_t_float16_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x1c211): undefined reference to `Qwen2LLM<signed char, float16_t, Qwen3MOE<signed char, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), decoder_Qwen3MOELLM_nf4x2_t_int8_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x1c511): undefined reference to `Qwen2LLM<nf4x2_t, signed char, Qwen3MOE<nf4x2_t, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), hybridModel_Qwen3MOELLM_bfloat16_t_float16_t_float16_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x20e7f): undefined reference to `Qwen2LLM<bfloat16_t, float16_t, Qwen3MOE<bfloat16_t, bfloat16_t, bfloat16_t, bfloat16_t> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: models.cpp:(.text+0x20f22): undefined reference to `Qwen2LLM<float16_t, float16_t, Qwen3MOE<float16_t, float16_t, float16_t, float16_t> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), hybridModel_Qwen3MOELLM_bfloat16_t_uint4x2_t_int8_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x2183f): undefined reference to `Qwen2LLM<bfloat16_t, signed char, Qwen3MOE<bfloat16_t, bfloat16_t, bfloat16_t, bfloat16_t> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: models.cpp:(.text+0x218e2): undefined reference to `Qwen2LLM<uint4x2_t, signed char, Qwen3MOE<uint4x2_t, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), hybridModel_Qwen3MOELLM_w8a8_t_int8_t_float16_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x21e1f): undefined reference to `Qwen2LLM<w8a8, float16_t, Qwen3MOE<w8a8, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: models.cpp:(.text+0x21ec2): undefined reference to `Qwen2LLM<signed char, float16_t, Qwen3MOE<signed char, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), hybridModel_Qwen3MOELLM_bfloat16_t_int8_t_float16_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x2201f): undefined reference to `Qwen2LLM<bfloat16_t, float16_t, Qwen3MOE<bfloat16_t, bfloat16_t, bfloat16_t, bfloat16_t> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: models.cpp:(.text+0x220c2): undefined reference to `Qwen2LLM<signed char, float16_t, Qwen3MOE<signed char, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), hybridModel_Qwen3MOELLM_bfloat16_t_uint4x2_t_float16_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x225ff): undefined reference to `Qwen2LLM<bfloat16_t, float16_t, Qwen3MOE<bfloat16_t, bfloat16_t, bfloat16_t, bfloat16_t> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: models.cpp:(.text+0x226a2): undefined reference to `Qwen2LLM<uint4x2_t, float16_t, Qwen3MOE<uint4x2_t, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), hybridModel_Qwen3MOELLM_bfloat16_t_w8a8_t_int8_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x231af): undefined reference to `Qwen2LLM<bfloat16_t, signed char, Qwen3MOE<bfloat16_t, bfloat16_t, bfloat16_t, bfloat16_t> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: models.cpp:(.text+0x23252): undefined reference to `Qwen2LLM<w8a8, signed char, Qwen3MOE<w8a8, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), hybridModel_Qwen3MOELLM_w8a8_t_int8_t_int8_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x2378f): undefined reference to `Qwen2LLM<w8a8, signed char, Qwen3MOE<w8a8, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: models.cpp:(.text+0x23832): undefined reference to `Qwen2LLM<signed char, signed char, Qwen3MOE<signed char, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), hybridModel_Qwen3MOELLM_w8a8_t_uint4x2_t_float16_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x23b7f): undefined reference to `Qwen2LLM<w8a8, float16_t, Qwen3MOE<w8a8, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: models.cpp:(.text+0x23c22): undefined reference to `Qwen2LLM<uint4x2_t, float16_t, Qwen3MOE<uint4x2_t, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), hybridModel_Qwen3MOELLM_bfloat16_t_nf4x2_t_float16_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x2453f): undefined reference to `Qwen2LLM<bfloat16_t, float16_t, Qwen3MOE<bfloat16_t, bfloat16_t, bfloat16_t, bfloat16_t> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: models.cpp:(.text+0x245e2): undefined reference to `Qwen2LLM<nf4x2_t, float16_t, Qwen3MOE<nf4x2_t, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), hybridModel_Qwen3MOELLM_bfloat16_t_nf4x2_t_int8_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x2473f): undefined reference to `Qwen2LLM<bfloat16_t, signed char, Qwen3MOE<bfloat16_t, bfloat16_t, bfloat16_t, bfloat16_t> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: models.cpp:(.text+0x247e2): undefined reference to `Qwen2LLM<nf4x2_t, signed char, Qwen3MOE<nf4x2_t, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), hybridModel_Qwen3MOELLM_bfloat16_t_int8_t_int8_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x250ff): undefined reference to `Qwen2LLM<bfloat16_t, signed char, Qwen3MOE<bfloat16_t, bfloat16_t, bfloat16_t, bfloat16_t> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: models.cpp:(.text+0x251a2): undefined reference to `Qwen2LLM<signed char, signed char, Qwen3MOE<signed char, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), hybridModel_Qwen3MOELLM_bfloat16_t_float16_t_int8_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x254ef): undefined reference to `Qwen2LLM<bfloat16_t, signed char, Qwen3MOE<bfloat16_t, bfloat16_t, bfloat16_t, bfloat16_t> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: models.cpp:(.text+0x25592): undefined reference to `Qwen2LLM<float16_t, signed char, Qwen3MOE<float16_t, float16_t, float16_t, float16_t> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), hybridModel_Qwen3MOELLM_w8a8_t_uint4x2_t_int8_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x2628f): undefined reference to `Qwen2LLM<w8a8, signed char, Qwen3MOE<w8a8, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: models.cpp:(.text+0x26332): undefined reference to `Qwen2LLM<uint4x2_t, signed char, Qwen3MOE<uint4x2_t, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), hybridModel_Qwen3MOELLM_bfloat16_t_w8a8_t_float16_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x26c4f): undefined reference to `Qwen2LLM<bfloat16_t, float16_t, Qwen3MOE<bfloat16_t, bfloat16_t, bfloat16_t, bfloat16_t> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: models.cpp:(.text+0x26cf2): undefined reference to `Qwen2LLM<w8a8, float16_t, Qwen3MOE<w8a8, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), hybridModel_Qwen3MOELLM_w8a8_t_nf4x2_t_float16_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x2722f): undefined reference to `Qwen2LLM<w8a8, float16_t, Qwen3MOE<w8a8, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: models.cpp:(.text+0x272d2): undefined reference to `Qwen2LLM<nf4x2_t, float16_t, Qwen3MOE<nf4x2_t, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), hybridModel_Qwen3MOELLM_w8a8_t_nf4x2_t_int8_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x27fcf): undefined reference to `Qwen2LLM<w8a8, signed char, Qwen3MOE<w8a8, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: models.cpp:(.text+0x28072): undefined reference to `Qwen2LLM<nf4x2_t, signed char, Qwen3MOE<nf4x2_t, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), decoder_Qwen3MOELLM_nf4x2_t_float16_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x1a291): undefined reference to `Qwen2LLM<nf4x2_t, float16_t, Qwen3MOE<nf4x2_t, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), decoder_Qwen3MOELLM_int8_t_int8_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x1a351): undefined reference to `Qwen2LLM<signed char, signed char, Qwen3MOE<signed char, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), decoder_Qwen3MOELLM_bfloat16_t_int8_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x1a651): undefined reference to `Qwen2LLM<bfloat16_t, signed char, Qwen3MOE<bfloat16_t, bfloat16_t, bfloat16_t, bfloat16_t> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), decoder_Qwen3MOELLM_bfloat16_t_float16_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x1a7d1): undefined reference to `Qwen2LLM<bfloat16_t, float16_t, Qwen3MOE<bfloat16_t, bfloat16_t, bfloat16_t, bfloat16_t> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), decoder_Qwen3MOELLM_uint4x2_t_float16_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x1a951): undefined reference to `Qwen2LLM<uint4x2_t, float16_t, Qwen3MOE<uint4x2_t, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), decoder_Qwen3MOELLM_float16_t_int8_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x1aa11): undefined reference to `Qwen2LLM<float16_t, signed char, Qwen3MOE<float16_t, float16_t, float16_t, float16_t> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), decoder_Qwen3MOELLM_float16_t_float16_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x1ab91): undefined reference to `Qwen2LLM<float16_t, float16_t, Qwen3MOE<float16_t, float16_t, float16_t, float16_t> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), decoder_Qwen3MOELLM_uint4x2_t_int8_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x1add1): undefined reference to `Qwen2LLM<uint4x2_t, signed char, Qwen3MOE<uint4x2_t, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), decoder_Qwen3MOELLM_w8a8_t_int8_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x1bfd1): undefined reference to `Qwen2LLM<w8a8, signed char, Qwen3MOE<w8a8, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), decoder_Qwen3MOELLM_w8a8_t_float16_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x1c151): undefined reference to `Qwen2LLM<w8a8, float16_t, Qwen3MOE<w8a8, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), decoder_Qwen3MOELLM_int8_t_float16_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x1c211): undefined reference to `Qwen2LLM<signed char, float16_t, Qwen3MOE<signed char, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), decoder_Qwen3MOELLM_nf4x2_t_int8_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x1c511): undefined reference to `Qwen2LLM<nf4x2_t, signed char, Qwen3MOE<nf4x2_t, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), hybridModel_Qwen3MOELLM_bfloat16_t_float16_t_float16_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x20e7f): undefined reference to `Qwen2LLM<bfloat16_t, float16_t, Qwen3MOE<bfloat16_t, bfloat16_t, bfloat16_t, bfloat16_t> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: models.cpp:(.text+0x20f22): undefined reference to `Qwen2LLM<float16_t, float16_t, Qwen3MOE<float16_t, float16_t, float16_t, float16_t> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), hybridModel_Qwen3MOELLM_bfloat16_t_uint4x2_t_int8_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x2183f): undefined reference to `Qwen2LLM<bfloat16_t, signed char, Qwen3MOE<bfloat16_t, bfloat16_t, bfloat16_t, bfloat16_t> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: models.cpp:(.text+0x218e2): undefined reference to `Qwen2LLM<uint4x2_t, signed char, Qwen3MOE<uint4x2_t, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), hybridModel_Qwen3MOELLM_w8a8_t_int8_t_float16_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x21e1f): undefined reference to `Qwen2LLM<w8a8, float16_t, Qwen3MOE<w8a8, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: models.cpp:(.text+0x21ec2): undefined reference to `Qwen2LLM<signed char, float16_t, Qwen3MOE<signed char, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), hybridModel_Qwen3MOELLM_bfloat16_t_int8_t_float16_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x2201f): undefined reference to `Qwen2LLM<bfloat16_t, float16_t, Qwen3MOE<bfloat16_t, bfloat16_t, bfloat16_t, bfloat16_t> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: models.cpp:(.text+0x220c2): undefined reference to `Qwen2LLM<signed char, float16_t, Qwen3MOE<signed char, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), hybridModel_Qwen3MOELLM_bfloat16_t_uint4x2_t_float16_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x225ff): undefined reference to `Qwen2LLM<bfloat16_t, float16_t, Qwen3MOE<bfloat16_t, bfloat16_t, bfloat16_t, bfloat16_t> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: models.cpp:(.text+0x226a2): undefined reference to `Qwen2LLM<uint4x2_t, float16_t, Qwen3MOE<uint4x2_t, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), hybridModel_Qwen3MOELLM_bfloat16_t_w8a8_t_int8_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x231af): undefined reference to `Qwen2LLM<bfloat16_t, signed char, Qwen3MOE<bfloat16_t, bfloat16_t, bfloat16_t, bfloat16_t> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: models.cpp:(.text+0x23252): undefined reference to `Qwen2LLM<w8a8, signed char, Qwen3MOE<w8a8, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), hybridModel_Qwen3MOELLM_w8a8_t_int8_t_int8_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x2378f): undefined reference to `Qwen2LLM<w8a8, signed char, Qwen3MOE<w8a8, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: models.cpp:(.text+0x23832): undefined reference to `Qwen2LLM<signed char, signed char, Qwen3MOE<signed char, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), hybridModel_Qwen3MOELLM_w8a8_t_uint4x2_t_float16_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x23b7f): undefined reference to `Qwen2LLM<w8a8, float16_t, Qwen3MOE<w8a8, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: models.cpp:(.text+0x23c22): undefined reference to `Qwen2LLM<uint4x2_t, float16_t, Qwen3MOE<uint4x2_t, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), hybridModel_Qwen3MOELLM_bfloat16_t_nf4x2_t_float16_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x2453f): undefined reference to `Qwen2LLM<bfloat16_t, float16_t, Qwen3MOE<bfloat16_t, bfloat16_t, bfloat16_t, bfloat16_t> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: models.cpp:(.text+0x245e2): undefined reference to `Qwen2LLM<nf4x2_t, float16_t, Qwen3MOE<nf4x2_t, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), hybridModel_Qwen3MOELLM_bfloat16_t_nf4x2_t_int8_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x2473f): undefined reference to `Qwen2LLM<bfloat16_t, signed char, Qwen3MOE<bfloat16_t, bfloat16_t, bfloat16_t, bfloat16_t> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: models.cpp:(.text+0x247e2): undefined reference to `Qwen2LLM<nf4x2_t, signed char, Qwen3MOE<nf4x2_t, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), hybridModel_Qwen3MOELLM_bfloat16_t_int8_t_int8_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x250ff): undefined reference to `Qwen2LLM<bfloat16_t, signed char, Qwen3MOE<bfloat16_t, bfloat16_t, bfloat16_t, bfloat16_t> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: models.cpp:(.text+0x251a2): undefined reference to `Qwen2LLM<signed char, signed char, Qwen3MOE<signed char, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), hybridModel_Qwen3MOELLM_bfloat16_t_float16_t_int8_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x254ef): undefined reference to `Qwen2LLM<bfloat16_t, signed char, Qwen3MOE<bfloat16_t, bfloat16_t, bfloat16_t, bfloat16_t> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: models.cpp:(.text+0x25592): undefined reference to `Qwen2LLM<float16_t, signed char, Qwen3MOE<float16_t, float16_t, float16_t, float16_t> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), hybridModel_Qwen3MOELLM_w8a8_t_uint4x2_t_int8_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x2628f): undefined reference to `Qwen2LLM<w8a8, signed char, Qwen3MOE<w8a8, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: models.cpp:(.text+0x26332): undefined reference to `Qwen2LLM<uint4x2_t, signed char, Qwen3MOE<uint4x2_t, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), hybridModel_Qwen3MOELLM_bfloat16_t_w8a8_t_float16_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x26c4f): undefined reference to `Qwen2LLM<bfloat16_t, float16_t, Qwen3MOE<bfloat16_t, bfloat16_t, bfloat16_t, bfloat16_t> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: models.cpp:(.text+0x26cf2): undefined reference to `Qwen2LLM<w8a8, float16_t, Qwen3MOE<w8a8, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), hybridModel_Qwen3MOELLM_w8a8_t_nf4x2_t_float16_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x2722f): undefined reference to `Qwen2LLM<w8a8, float16_t, Qwen3MOE<w8a8, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: models.cpp:(.text+0x272d2): undefined reference to `Qwen2LLM<nf4x2_t, float16_t, Qwen3MOE<nf4x2_t, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: ../../src/libxfastertransformer_static.a(models.cpp.o): in function `std::_Function_handler<AbstractDecoder* (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), hybridModel_Qwen3MOELLM_w8a8_t_nf4x2_t_int8_t::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
models.cpp:(.text+0x27fcf): undefined reference to `Qwen2LLM<w8a8, signed char, Qwen3MOE<w8a8, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: models.cpp:(.text+0x28072): undefined reference to `Qwen2LLM<nf4x2_t, signed char, Qwen3MOE<nf4x2_t, float, float, float> >::Qwen2LLM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
collect2: error: ld returned 1 exit status
make[2]: *** [examples/cpp/CMakeFiles/example.dir/build.make:130: example] Error 1
make[1]: *** [CMakeFiles/Makefile2:660: examples/cpp/CMakeFiles/example.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
collect2: error: ld returned 1 exit status
make[2]: *** [examples/cpp/CMakeFiles/cb_check.dir/build.make:130: cb_check] Error 1
make[1]: *** [CMakeFiles/Makefile2:693: examples/cpp/CMakeFiles/cb_check.dir/all] Error 2
make: *** [Makefile:91: all] Error 2

Jasonjunsu avatar May 23 '25 06:05 Jasonjunsu

@Duyi-Wang @changqi1

Jasonjunsu avatar May 23 '25 06:05 Jasonjunsu

@Jasonjunsu update torch to 2.7.0

abenmao avatar May 24 '25 01:05 abenmao

@Jasonjunsu update torch to 2.7.0

My torch version has always been 2.7.0. I think maybe is has sth with CMakeLists.txt, I've made changes as below. but it still doesn't work

~/repo/xFasterTransformer/build$ git diff
diff --git a/CMakeLists.txt b/CMakeLists.txt
index 5b286eb..785142b 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -179,12 +179,12 @@ if(WITH_GPU)
 else()
     # Enable AVX512_FP16 optimization
     # add_definitions(-DAVX512_FP32_WEIGHT_ONLY_FP16=true)
-    add_definitions(-DAVX512_FP16_WEIGHT_ONLY_FP16=true)
+    # add_definitions(-DAVX512_FP16_WEIGHT_ONLY_FP16=true)
     add_definitions(-DAVX512_BF16_WEIGHT_ONLY_BF16=true)
-    # add_definitions(-DAVX512_FP32_WEIGHT_ONLY_INT8=true)
-    add_definitions(-DAVX512_FP16_WEIGHT_ONLY_INT8=true)
-    # add_definitions(-DAVX512_FP32_WEIGHT_ONLY_INT4=true)
-    add_definitions(-DAVX512_FP16_WEIGHT_ONLY_INT4=true)
+    add_definitions(-DAVX512_FP32_WEIGHT_ONLY_INT8=true)
+    # add_definitions(-DAVX512_FP16_WEIGHT_ONLY_INT8=true)
+    add_definitions(-DAVX512_FP32_WEIGHT_ONLY_INT4=true)
+    # add_definitions(-DAVX512_FP16_WEIGHT_ONLY_INT4=true)
     add_definitions(-DAVX512_FP32_WEIGHT_ONLY_NF4=true)
     # add_definitions(-DAVX512_FP16_WEIGHT_ONLY_NF4=true)
     # Enable AMX_FP16 optimization

This is my cpu info

 lscpu
Architecture:             x86_64
  CPU op-mode(s):         32-bit, 64-bit
  Address sizes:          46 bits physical, 57 bits virtual
  Byte Order:             Little Endian
CPU(s):                   128
  On-line CPU(s) list:    0-127
Vendor ID:                GenuineIntel
  Model name:             Intel(R) Xeon(R) Platinum 8362 CPU @ 2.80GHz
    CPU family:           6
    Model:                106
    Thread(s) per core:   2
    Core(s) per socket:   32
    Socket(s):            2
    Stepping:             6
    BogoMIPS:             5600.00
    Flags:                fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs
                           bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt
                          tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 intel_ppin ssbd mba ibrs ibpb stibp ibrs_enhanced tpr_shadow flexpriority ept vpid ept_ad
                           fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm rdt_a avx512f avx512dq rdseed adx smap avx512ifma clflushopt clwb intel_pt avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec
                          xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local split_lock_detect wbnoinvd dtherm ida arat pln pts vnmi avx512vbmi umip pku ospke avx512_vbmi2 gfni vaes vpclmulqdq av
                          x512_vnni avx512_bitalg tme avx512_vpopcntdq la57 rdpid fsrm md_clear pconfig flush_l1d arch_capabilities

Jasonjunsu avatar May 26 '25 01:05 Jasonjunsu

Hi! I also encounter this problem. Did you get any solution?

HaibinLai avatar Jun 06 '25 04:06 HaibinLai

@Jasonjunsu INT4 is currently not recommended. @HaibinLai you can upgrade gcc to 13 or higher.

abenmao avatar Jun 06 '25 05:06 abenmao

Hi! I found that adding "-DBUILD_WITH_SHARED_LIBS=ON" maybe work cmake -DBUILD_WITH_SHARED_LIBS=ON -DCMAKE_BUILD_TYPE=Debug .. make -j

HaibinLai avatar Jun 06 '25 05:06 HaibinLai