TensorRT-LLM icon indicating copy to clipboard operation
TensorRT-LLM copied to clipboard
verified publisher icon NVIDIA

when will the high-level-api support qwen model？

Open thend-wk opened this issue 1 year ago • 1 comments

thend-wk avatar Apr 30 '24 01:04 thend-wk

when i'm running following script: python3 llm_examples.py --task run_llm_from_huggingface_model \ --prompt="你是谁？" \ --tp_size=8 \ --dump_engine_dir /docker_storage/trtModels/fp16/8-gpu/Qwen1.5-72B-Chat \ --hf_model_dir=/docker_storage/Qwen1.5-72B-Chat KeyError: 'Unsupported model architecture: Qwen2ForCausalLM, only LlamaForCausalLM, MixtralForCausalLM are supported now.'

thend-wk avatar Apr 30 '24 08:04 thend-wk