text-generation-webui icon indicating copy to clipboard operation
text-generation-webui copied to clipboard

Published 20 hours ago • verified publisher icon oobabooga

Support GLM-10b Model?

Open Ph0rk0z opened this issue 1 year ago • 6 comments

Here: https://huggingface.co/THUDM/glm-10b

They have some external code that has to run, which I enabled but it still errors. Is this a different type of model that can't load by CasualLLM?

I only ask because it's 9-11b is the sweet spot for 24gb without 8bit or any other tricks.

There is also https://huggingface.co/hyunwoongko/reddit-9B which looks like it would make a great chatbot but uses a different pytorch function. I can get it to load but not generate. Their model card certainly wins me over.

Ph0rk0z avatar Mar 03 '23 19:03 Ph0rk0z

Correct me if I am wrong, but it seems to be a model for filling gaps like BERT rather than a model that predicts the next token for open-ended generation.

oobabooga avatar Mar 03 '23 20:03 oobabooga

@oobabooga It's trained to do both. At least the 130b version is.

ogkalu2 avatar Mar 04 '23 14:03 ogkalu2

https://github.com/THUDM/ChatGLM-6B seems they have a model for chat now

npk48 avatar Mar 14 '23 07:03 npk48

Honestly ... even at 6b this model is impressive... far better than pygmalion....

Stonedge avatar Apr 15 '23 06:04 Stonedge

This issue has been closed due to inactivity for 6 weeks. If you believe it is still relevant, please leave a comment below. You can tag a developer in your comment.

github-actions[bot] avatar Dec 08 '23 23:12 github-actions[bot]