Badis

I got this error when trying to convert this alpaca 4bit file: https://huggingface.co/ozcur/alpaca-native-4bit/tree/main (I renamed the alpaca7b-4bit.pt to llama7b-4bit.pt to match your command lines) ``` D:\Large Language Models\CONVERTISSEURS\gptq to ggml>python...

> > I guess this converter won't work on models that aren't the llama raw model right? > > I haven't tested it with any other models but I'd like...

I'm not sure it's a good idea to remove this, when we'll use llama.ccp on a front end, we could use this \end{code} as a flag to close a panel....

@ggerganov Loras are used a lot in Stable Diffusion and in the webui version of llama aswell https://github.com/oobabooga/text-generation-webui/issues/332 (it doesn't work for the 4 bits for them atm though)

> Please review the issue reporting guidelines in #239 and provide a better description of the issue you are observing. I added more details based on your guideline, I hope...

@PriNova I see, thanks for your answer I learned something today! But still I can see a space at the begining of the text, I think I hadn't that before,...

@Ronsor I used your script and it looks like it did actually add the token on the tokenizer.model But now I have a new error... looks like the issue is...

On the ```convert_q4(src_name, dst_name, permute=False):``` function I changed: ``` zeros = model[f"{src_name}.zeros"].numpy() ... qweight = model[f"{src_name}.weight"].numpy().T # transpose ``` to ``` zeros = model[f"{src_name}.qzeros"].numpy() ... qweight = model[f"{src_name}.qweight"].numpy().T # transpose...

@comex I'm not sure it was a good idea to convert your addends and scales into int32, those tensors have really small numbers and we're loosing all the informations like...

oobabooga merged a PR that makes the alpaca-7b-4bit-GPTQ-native works now https://github.com/oobabooga/text-generation-webui/commit/49c10c5570b595e9d4fdcb496c456a9982ede070 That's funny it worked because it uses the exact same tokenizer model (the one with 32000 token) even though...