llama.cpp
llama.cpp copied to clipboard
Bug: Quantizing Llama 3.1 70B to Q4_K_S with imatrix gives NaN
What happened?
Trying to quantize Llama 3.1 70B to Q4_K_S with imatrix gives NaN for block 48
Tagging @slaren because you always seem to solve these
Didn't see it yet on any other quant size
Name and Version
b3441
What operating system are you seeing the problem on?
Linux
Relevant log output
ggml_validate_row_data: found nan value at block 48