Sigbjørn Skjæret

@BarfingLemurs That makes no sense, why would the imatrix need to be changed? Anyway, tried again with #5999 and it's no longer spouting gibberish, however it's still worse than before,...

@ikawrakow Any feedback appreciated, I can provide you with whatever you need to help figure this out.

@ikawrakow All DeepSeek models require --pad-vocab but I had no problems calculating an imatrix, in fact just tried again with the latest build and still works fine, so that's pretty...

Just to make sure nothing else is broken I also quickly requantized IQ2_XXS with the latest build and tested it, works perfectly: ``` get_current_weather(location="Oslo") ```

@ikawrakow That's what's so weird, why is it only affecting IQ1_S? As I said, all other quants are working fine, even after requantizing with latest build. I've even made sure...

@ikawrakow Now we're getting somewhere, I first tried just regenerating the imatrix the same way I did originally (just to make sure there was nothing wrong with is, as suggested...

@ikawrakow I've been digging through the IQ1_S quantizing functions and made the following changes that seems to fix the problem: ```diff diff --git a/ggml-quants.c b/ggml-quants.c index 06665eb2..936f9122 100644 --- a/ggml-quants.c...

@ikawrakow It's probably best that you run the tests to ensure all the variables are the same (and that I haven't made a mistake). I can wait. :)

@hyperbolic-c Ah, I remember I also had to use `--vocab-type bpe` when converting otherwise it would choose the wrong tokenizer.

@hyperbolic-c Did you try again after converting with the right tokenizer? It worked for me, and for @ikawrakow when using my converted GGUF. If it still doesn't work for you,...