SIGSEGV during inference

[ko-alex]

Name and Version

(gguf) ➜ llama.cpp git:(master) ./build/bin/llama-cli --version ggml_cuda_init: GGML_CUDA_FORCE_MMQ: yes ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no ggml_cuda_init: found 2 CUDA devices: Device 0: NVIDIA GeForce RTX 4060 Ti, compute capability 8.9, VMM: yes Device 1: NVIDIA GeForce RTX 3060, compute capability 8.6, VMM: yes version: 4564 (acd38efe) built with cc (Debian 12.2.0-14) 12.2.0 for x86_64-linux-gnu

Operating systems

Linux

Which llama.cpp modules do you know to be affected?

llama-cli

Command line

./build/bin/llama-cli -t 6 --color --interactive --conversation --multiline-input --mirostat 2 --ctx-size 16384 --keep -1 --flash-attn --repeat-penalty 1.2 --n-gpu-layers 44 --temp 0.3 --cache-type-v q8_0 --cache-type-k q8_0

Problem description & steps to reproduce

Suspect: '--cache-type-v q8_0 --cache-type-k q8_0' might be responsible.

Problem: during the chat, eventually this happens (output from cli, no further details): [1] 12457 segmentation fault (core dumped) ./build/bin/llama-cli

$ coredumpctl list Mon 2025-01-27 20:48:06 IST 12457 1000 1000 SIGSEGV present /work/src/llama.cpp/build/bin/llama-cli 1.2G

From core dump, call stack:

Stack trace of thread 12474: #0 0x00007f128f771aab n/a (libc.so.6 + 0x152aab) #1 0x00007f128f5c68af ggml_graph_compute_thread.isra.0 (libggml-cpu.so + 0x5a8af) #2 0x00007f128f540d8e n/a (libgomp.so.1 + 0x1cd8e) #3 0x00007f128f6a81c4 n/a (libc.so.6 + 0x891c4) #4 0x00007f128f72885c n/a (libc.so.6 + 0x10985c)

Stack trace of thread 12472: #0 0x00007f128f771ae9 n/a (libc.so.6 + 0x152ae9) #1 0x00007f128f5c68af ggml_graph_compute_thread.isra.0 (libggml-cpu.so + 0x5a8af) #2 0x00007f128f540d8e n/a (libgomp.so.1 + 0x1cd8e) #3 0x00007f128f6a81c4 n/a (libc.so.6 + 0x891c4) #4 0x00007f128f72885c n/a (libc.so.6 + 0x10985c)

Stack trace of thread 12470: #0 0x00007f128f771b00 n/a (libc.so.6 + 0x152b00) #1 0x00007f128f5c68af ggml_graph_compute_thread.isra.0 (libggml-cpu.so + 0x5a8af) #2 0x00007f128f540d8e n/a (libgomp.so.1 + 0x1cd8e) #3 0x00007f128f6a81c4 n/a (libc.so.6 + 0x891c4) #4 0x00007f128f72885c n/a (libc.so.6 + 0x10985c)

Stack trace of thread 12471: #0 0x00007f128f771af3 n/a (libc.so.6 + 0x152af3) #1 0x00007f128f5c68af ggml_graph_compute_thread.isra.0 (libggml-cpu.so + 0x5a8af) #2 0x00007f128f540d8e n/a (libgomp.so.1 + 0x1cd8e) #3 0x00007f128f6a81c4 n/a (libc.so.6 + 0x891c4) #4 0x00007f128f72885c n/a (libc.so.6 + 0x10985c)

Stack trace of thread 12473: #0 0x00007f128f771ae4 n/a (libc.so.6 + 0x152ae4) #1 0x00007f128f5c68af ggml_graph_compute_thread.isra.0 (libggml-cpu.so + 0x5a8af) #2 0x00007f128f540d8e n/a (libgomp.so.1 + 0x1cd8e) #3 0x00007f128f6a81c4 n/a (libc.so.6 + 0x891c4) #4 0x00007f128f72885c n/a (libc.so.6 + 0x10985c)

Stack trace of thread 12466: #0 0x00007f128f6a4f16 n/a (libc.so.6 + 0x85f16) #1 0x00007f128f6a78bc pthread_cond_timedwait (libc.so.6 + 0x888bc) #2 0x00007f127ebcac8a n/a (libcuda.so.1 + 0x1cac8a) #3 0x00007f127ec6dee3 n/a (libcuda.so.1 + 0x26dee3) #4 0x00007f128f6a81c4 n/a (libc.so.6 + 0x891c4) #5 0x00007f128f72885c n/a (libc.so.6 + 0x10985c)

Stack trace of thread 12457: #0 0x00007f128f771ae9 n/a (libc.so.6 + 0x152ae9) #1 0x00007f128f5c68af ggml_graph_compute_thread.isra.0 (libggml-cpu.so + 0x5a8af) #2 0x00007f128f5380b6 GOMP_parallel (libgomp.so.1 + 0x140b6) #3 0x00007f128f597a5c ggml_graph_compute (libggml-cpu.so + 0x2ba5c) #4 0x00007f128f5a61c2 _ZL30ggml_backend_cpu_graph_computeP12ggml_backendP11ggml_cgraph (libggml-cpu.so + 0x3a1c2) #5 0x00007f128fb67f83 ggml_backend_sched_graph_compute_async (libggml-base.so + 0x26f83) #6 0x00007f128fc694b0 _ZL19llama_graph_computeR13llama_contextP11ggml_cgraphiP15ggml_threadpool (libllama.so + 0x4e4b0) #7 0x00007f128fc6d8f3 llama_kv_cache_update (libllama.so + 0x528f3) #8 0x00007f128fc6e87e _ZL17llama_decode_implR13llama_context11llama_batch (libllama.so + 0x5387e) #9 0x00007f128fc6fa87 llama_decode (libllama.so + 0x54a87) #10 0x000055df853f5596 main (llama-cli + 0x22596) #11 0x00007f128f64624a n/a (libc.so.6 + 0x2724a) #12 0x00007f128f646305 __libc_start_main (libc.so.6 + 0x27305) #13 0x000055df853f95d1 _start (llama-cli + 0x265d1)

Stack trace of thread 12467: #0 0x00007f128f6a4f16 n/a (libc.so.6 + 0x85f16) #1 0x00007f128f6a75d8 pthread_cond_wait (libc.so.6 + 0x885d8) #2 0x000055df854b957b _ZZN10common_log6resumeEvENKUlvE_clEv (llama-cli + 0xe657b) #3 0x00007f128f8d44a3 n/a (libstdc++.so.6 + 0xd44a3) #4 0x00007f128f6a81c4 n/a (libc.so.6 + 0x891c4) #5 0x00007f128f72885c n/a (libc.so.6 + 0x10985c)

Stack trace of thread 12469: #0 0x00007f128f6a4f16 n/a (libc.so.6 + 0x85f16) #1 0x00007f128f6a78bc pthread_cond_timedwait (libc.so.6 + 0x888bc) #2 0x00007f127ebcac8a n/a (libcuda.so.1 + 0x1cac8a) #3 0x00007f127ec6dee3 n/a (libcuda.so.1 + 0x26dee3) #4 0x00007f128f6a81c4 n/a (libc.so.6 + 0x891c4) #5 0x00007f128f72885c n/a (libc.so.6 + 0x10985c)

Stack trace of thread 12465: #0 0x00007f128f71b1df __poll (libc.so.6 + 0xfc1df) #1 0x00007f127ec761ef n/a (libcuda.so.1 + 0x2761ef) #2 0x00007f127ed3a67f n/a (libcuda.so.1 + 0x33a67f) #3 0x00007f127ec6dee3 n/a (libcuda.so.1 + 0x26dee3) #4 0x00007f128f6a81c4 n/a (libc.so.6 + 0x891c4) #5 0x00007f128f72885c n/a (libc.so.6 + 0x10985c)

Stack trace of thread 12458: #0 0x00007f128f71b1df __poll (libc.so.6 + 0xfc1df) #1 0x00007f127ec761ef n/a (libcuda.so.1 + 0x2761ef) #2 0x00007f127ed3a67f n/a (libcuda.so.1 + 0x33a67f) #3 0x00007f127ec6dee3 n/a (libcuda.so.1 + 0x26dee3) #4 0x00007f128f6a81c4 n/a (libc.so.6 + 0x891c4) #5 0x00007f128f72885c n/a (libc.so.6 + 0x10985c)

Stack trace of thread 12468: #0 0x00007f128f71b1df __poll (libc.so.6 + 0xfc1df) #1 0x00007f127ec761ef n/a (libcuda.so.1 + 0x2761ef) #2 0x00007f127ed3a67f n/a (libcuda.so.1 + 0x33a67f) #3 0x00007f127ec6dee3 n/a (libcuda.so.1 + 0x26dee3) #4 0x00007f128f6a81c4 n/a (libc.so.6 + 0x891c4) #5 0x00007f128f72885c n/a (libc.so.6 + 0x10985c) ELF object binary architecture: AMD x86-64

First Bad Commit

No response

Relevant log output