llama.cpp icon indicating copy to clipboard operation
llama.cpp copied to clipboard

Bug: Quantizing Llama 3.1 70B to Q4_K_S with imatrix gives NaN

Open bartowski1182 opened this issue 7 months ago • 3 comments

What happened?

Trying to quantize Llama 3.1 70B to Q4_K_S with imatrix gives NaN for block 48

Tagging @slaren because you always seem to solve these

Didn't see it yet on any other quant size

Name and Version

b3441

What operating system are you seeing the problem on?

Linux

Relevant log output

ggml_validate_row_data: found nan value at block 48

bartowski1182 avatar Jul 23 '24 23:07 bartowski1182