Native support of ChatGLM-6b

[xianbaoqian]

Feature request

Support https://huggingface.co/THUDM/chatglm-6b (and its int4 variants) in the Transformers library instead of relying on remote code execution.

Motivation

This model performs really well (despite being a small model compared to large ones) and got a LOT of attention recently. It might be the SD moment for LLM IMO as it runs perfectly on consumer GPUs.

It would be great if Transformers can have native support for this model, instead of relying on remote code execution. A native integration will also make it much easier to use the model on Inference API / Endpoints.

Your contribution

cc @sgugger @osanseviero

[sgugger]

Transformers does have native support for it even if it's not in the lib itself ;-) I see this as a chance to better support models with code on the Hub since that is the way the author chose, and since it will be more and more the norm as we cannot have the library grow exponentially.

Of course, if the authors prefer to integrate the model in the library directly, we would be happy to look at the PR and help them merge it. We can also revisit if the issue gets a lot of traction and integrate it ourselves directly.

[LysandreJik]

I echo what Sylvain is saying above.

Additionally, for readers, if you would like this model to be integrated within the library nonetheless for it to be constantly tested and up-to-date with our API, please upvote the original post or add a comment mentioning so in this issue as this will help us identify models that should be more actively tested.

Thanks!

[xianbaoqian]

Thanks for all great inputs! Let's see how much demand we gathered for this one.

Just for your information ChatGLM-6b is the No. 1 model on the trending page now.

[Disastorm]

This model performs really well (despite being a small model compared to large ones) and got a LOT of attention recently. It might be the SD moment for LLM IMO as it runs perfectly on consumer GPUs.

It does seem quite good but for it to be the true SD moment I think the license would have to allow commercial use, which it doesn't.