支持更多的中文模型：Qwen、Baichuan、InternLM、ChatGLM2等多卡部署

[taishan1994]

Model description

首先，十分感谢您们的开源。

然后。以上都是比较有名的中文开源大模型，基本上都可以使用transformers库加载并进行推理。

最后。在单张卡上使用TGI进行推理是没有问题的，但是在多张卡上会报错shard is not supported for AutoModel。在受限的资源下，比如两张12G的显卡，使用多卡部署还是很有必要的，希望能够支持更多的中文模型。

Open source status

• [X] The model implementation is available
• [X] The model weights are available

Provide useful links for the implementation

https://github.com/InternLM/InternLM

https://github.com/QwenLM/Qwen-7B

https://github.com/baichuan-inc/Baichuan-7B https://github.com/baichuan-inc/Baichuan-13B

https://github.com/THUDM/ChatGLM2-6B

[mafamily2496]

千问-chat对话没法结束是啥情况？

[KelleyYin]

千问-chat对话没法结束是啥情况？

Do you add stop words when decoding ?

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[webdxq]

这是来自QQ邮箱的自动回复邮件。您好~您发送的邮件我已收到。谢谢您的邮件~

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.