LaaZa

> TypeError: qwen2_moe isn't supported yet. (base) > # Name Version Build Channel > > auto-gptq 0.7.1 pypi_0 pypi (base) Are you even using this PR? Also you need at...

> No, I haven't used this PR yet. I saw this branch and I'm very happy. Let's wait and merge it into the main branch. The speed of Qwen2moe is...

It is a GPTQ model, set wbits, groupsize and model_type to match the model. In this case wbits=4 and groupsize=128 and model_type llama. (some models are detected automatically) When the...

This is a bit of a problem, since `internlm` modeltype is already supported in AutoGPTQ but they are using completely different layout to normal InternLM. It would be possible to...

The model you are trying to use is sharded if it is the model directly from Qwen. AutoGPTQ does not yet support loading them. You can load them with Transformers(they...

I think this is a bit out of scope for AutoGPTQ as it is heavily based on CausalLM and even LLMs with vision are hit or miss if they will...

Are you sure the WizardLM instruction following is correct? It seems to start talking to itself after a while and almost completely ignoring me. It takes the role of the...

> > after a while > > Do you mean in follow-up questions? WizardLM does not have a concept of chat history with multiple questions. There's nothing to tag the...

It did happen to me without llava at some point but I don't remember what it might have been.

I think you just have to add the "Character" to your prompt manually. I mean that you manually manage the prompt since the prompt that you send is the only...