KeyError: 'ChatGLMForConditionalGeneration',glm4-9b,

[scutzhe]

System Info

Traceback (most recent call last): File "/usr/local/bin/trtllm-build", line 8, in sys.exit(main()) File "/usr/local/lib/python3.10/dist-packages/tensorrt_llm/commands/build.py", line 512, in main model_config = PretrainedConfig.from_json_file(config_path) File "/usr/local/lib/python3.10/dist-packages/tensorrt_llm/models/modeling_utils.py", line 304, in from_json_file return cls.from_dict(config) File "/usr/local/lib/python3.10/dist-packages/tensorrt_llm/models/modeling_utils.py", line 286, in from_dict model_cls = MODEL_MAP[config['architecture']] KeyError: 'ChatGLMForConditionalGeneration'

Who can help?

No response

Information

• [ ] The official example scripts
• [ ] My own modified scripts

Tasks

• [ ] An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• [ ] My own task or dataset (give details below)

Reproduction

trtllm-build --checkpoint_dir /opt/tritonserver/pth/LongWriter-glm4-9b-ckpt --gemm_plugin float16 --output_dir /opt/tritonserver/pth/LongWriter-glm4-9b-engine

Expected behavior

run well

actual behavior

see above

additional notes

no

[lfr-0531]

We can support the glm4-9b model, but cannot support the LongWriter-glm4-9b model now.

For glm4-9b model:

git clone https://huggingface.co/THUDM/glm-4-9b glm_4_9b
python3 convert_checkpoint.py --model_dir glm_4_9b --output_dir trt_ckpt/glm_4_9b/fp16/1-gpu
trtllm-build --checkpoint_dir trt_ckpt/glm_4_9b/fp16/1-gpu \
        --gemm_plugin float16 \
        --output_dir trt_engines/glm_4_9b/fp16/1-gpu

Mark this issue as a New Model Request.

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 15 days."

[AdamzNV]

As more and more new models enter the market, we have prepared comprehensive instructions for TRT-LLM developers on adapting to new models of interest. We encourage our community developers to expand the range of supported models, fostering an open ecosystem with rapid iterations.

Please try following these instructions and let us know if you encounter any issues during the adaptation process. We greatly appreciate your dedication.

[karljang]

@scutzhe , Apologies for the very delayed response. Is this ticket still relevant? If so, could you try the latest version to see if the issue persists?

[github-actions[bot]]

Issue has not received an update in over 14 days. Adding stale label.

[karljang]

Closing this issue as stale. If the issue persists in the latest release, please feel free to open a new one. Thank you!