AutoAWQ icon indicating copy to clipboard operation
AutoAWQ copied to clipboard
verified publisher icon casper-hansen

add support for telechat2

Open 1096125073 opened this issue 11 months ago • 2 comments

Two minor changes:

  1. Add support for the Telechat model (VLLM now supports the Telechat model).
  2. When executing the forward function, it is necessary to place the tensor in kwargs on a suitable device.

1096125073 avatar Jan 17 '25 10:01 1096125073

Hi, Telechat2 is a language model independently developed by China Telecom AI Company. In order to facilitate users to use the superior awq algorithm, this PR was raised.

1096125073 avatar Mar 19 '25 08:03 1096125073

https://huggingface.co/Tele-AI/TeleChat2-7B-32K

1096125073 avatar Mar 19 '25 09:03 1096125073