Bug: b3383 breaks Llama 3.1

[Azirine]

What happened?

b3383 (and above) breaks Llama 3.1.

./llama-cli -m Meta-Llama-3.1-8B-Instruct-Q8_0.gguf --no-mmap -fa -if --in-prefix "<|start_header_id|>user<|end_header_id|>\n\n" --in-suffix "<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n"

Name and Version

version: 3383 built with Apple clang version 15.0.0 (clang-1500.3.9.4) for arm64-apple-darwin23.5.0

What operating system are you seeing the problem on?

Mac

Relevant log output

**b3382**
main: interactive mode on.
Input prefix: '<|start_header_id|>user<|end_header_id|>

'
Input suffix: '<|eot_id|><|start_header_id|>assistant<|end_header_id|>

'
sampling: 
	repeat_last_n = 64, repeat_penalty = 1.000, frequency_penalty = 0.000, presence_penalty = 0.000
	top_k = 40, tfs_z = 1.000, top_p = 0.950, min_p = 0.050, typical_p = 1.000, temp = 0.800
	mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000
sampling order: 
CFG -> Penalties -> top_k -> tfs_z -> typical_p -> top_p -> min_p -> temperature 
generate: n_ctx = 131072, n_batch = 2048, n_predict = -1, n_keep = 0


== Running in interactive mode. ==
 - Press Ctrl+C to interject at any time.
 - Press Return to return control to the AI.
 - To return control without starting a new line, end your input with '/'.
 - If you want to submit another line, end your input with '\'.

<|start_header_id|>user<|end_header_id|>

**b3383**
error: input is empty

[Process completed]

[ko-alex]

workaround: add --prompt "hello"

[Azirine]

Fixed in new version https://huggingface.co/lmstudio-community/Meta-Llama-3.1-8B-Instruct-GGUF/commit/42de8c2795a4283d753795e84f5f7f1aa9fd267d

[Azirine]

Most quants are still affected, it was only fixed in the one above.

For example: https://huggingface.co/lmstudio-community/Meta-Llama-3.1-70B-Instruct-GGUF https://huggingface.co/legraphista/Meta-Llama-3.1-70B-Instruct-IMat-GGUF

[Azirine]

Fixed in new quants made after b3472 https://huggingface.co/legraphista/Meta-Llama-3.1-70B-Instruct-IMat-GGUF