Qwen2___5-0___5B-Instruct convert_checkpoint error

[giftyang]

System Info

x86_64 Intel(R) Xeon(R) Platinum 8378A CPU @ 3.00GHz Ubuntu 20.04.1 LTS NVIDIA A800-SXM4-80GB Driver Version: 550.54.15 CUDA Version: 12.1

Who can help?

No response

Information

• [X] The official example scripts
• [ ] My own modified scripts

Tasks

• [X] An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• [ ] My own task or dataset (give details below)

Reproduction

docker run a tensorrtllm_backend_v0.13.0 container in the container cd /app/tensorrt_llm/examples/qwen python3 convert_checkpoint.py --model_dir /data/qwen/Qwen2___5-0___5B-Instruct/
--output_dir /data/trtllm_models/tllm_checkpoint_1gpu_fp16_Qwen2___5-0___5B-Instruct
--tp_size 1
--dtype float16

Expected behavior

convert_checkpoint suscess

actual behavior

root@devserver:/app/tensorrt_llm/examples/qwen# python3 convert_checkpoint.py --model_dir /data/qwen/Qwen2___5-0___5B-Instruct/ \

                          --output_dir /data/trtllm_models/tllm_checkpoint_1gpu_fp16_Qwen2___5-0___5B-Instruct \
                          --tp_size 1 \
                          --dtype float16

[TensorRT-LLM] TensorRT-LLM version: 0.13.0 0.13.0 197it [00:00, 484.62it/s] Traceback (most recent call last): File "/app/tensorrt_llm/examples/qwen/convert_checkpoint.py", line 303, in main() File "/app/tensorrt_llm/examples/qwen/convert_checkpoint.py", line 295, in main convert_and_save_hf(args) File "/app/tensorrt_llm/examples/qwen/convert_checkpoint.py", line 251, in convert_and_save_hf execute(args.workers, [convert_and_save_rank] * world_size, args) File "/app/tensorrt_llm/examples/qwen/convert_checkpoint.py", line 258, in execute f(args, rank) File "/app/tensorrt_llm/examples/qwen/convert_checkpoint.py", line 241, in convert_and_save_rank qwen = QWenForCausalLM.from_hugging_face( File "/usr/local/lib/python3.10/dist-packages/tensorrt_llm/models/qwen/model.py", line 427, in from_hugging_face loader.generate_tllm_weights(model) File "/usr/local/lib/python3.10/dist-packages/tensorrt_llm/models/model_weights_loader.py", line 357, in generate_tllm_weights self.load(tllm_key, File "/usr/local/lib/python3.10/dist-packages/tensorrt_llm/models/model_weights_loader.py", line 278, in load v = sub_module.postprocess(tllm_key, v, **postprocess_kwargs) File "/usr/local/lib/python3.10/dist-packages/tensorrt_llm/layers/linear.py", line 391, in postprocess weights = weights.to(str_dtype_to_torch(self.dtype)) AttributeError: 'NoneType' object has no attribute 'to' Exception ignored in: <function PretrainedModel.del at 0x7faa60e40160>

additional notes