llama.cpp
llama.cpp copied to clipboard
Published
1 month ago
•
ggml-org
ggml-org
Eval bug: crash on faulty tool call
Name and Version
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no ggml_cuda_init: found 1 ROCm devices: Device 0: AMD Radeon RX 9070 XT, gfx1201 (0x1201), VMM: no, Wave Size: 32 version: 1032 (034e7aa) built with AMD clang version 19.0.0git (/srcdest/rocm-llvm d366fa84f3fdcbd4b10847ebd5db572ae12a34fb) for x86_64-pc-linux-gnu
Operating systems
Linux
GGML backends
HIP
Hardware
Ryzen 5 7600 + AMD RX 9070XT
Models
https://huggingface.co/unsloth/Qwen3-4B-Instruct-2507-GGUF
Problem description & steps to reproduce
this happened when my llm tried to make up an "example tool call". it makes llamacpp freeze up and often needs to be killed with a kill -9, though sometimes it lets itself be terminated normally
steps to reproduce:
- talk to your llm a bunch (i was using openwebui for it)
- somehow get it to generate a tool call for a nonexistent tool and with bad arguments
- watch llama-server crash
First Bad Commit
No response
Relevant log output
srv update_chat_: Parsing chat message: <think>
</think>
I apologize for the confusion earlier. Let me clarify how to properly format a tool call.
A valid tool call should look like this:
<tool_call>
{"name": "tool_name", "arguments": {"argument1": value1, "argument2": value2, ...}}
</tool_call>
For example, if I wanted to list the contents of your Downloads folder, I would format it like this:
<tool_call>
{"name": "tool_list_dir_post", "arguments": {"path": "/home/rosa/Downloads"}}
</tool_call>
Now, since you mentioned "shoopuf" and I can't find it, could you clarify what you're trying to do? Are you looking for a file, a note, or something else? Let me know how I can assist you properly!
Parsing input with format Hermes 2 Pro: <think>
</think>
I apologize for the confusion earlier. Let me clarify how to properly format a tool call.
A valid tool call should look like this:
<tool_call>
{"name": "tool_name", "arguments": {"argument1": value1, "argument2": value2, ...}}
</tool_call>
For example, if I wanted to list the contents of your Downloads folder, I would format it like this:
<tool_call>
{"name": "tool_list_dir_post", "arguments": {"path": "/home/rosa/Downloads"}}
</tool_call>
Now, since you mentioned "shoopuf" and I can't find it, could you clarify what you're trying to do? Are you looking for a file, a note, or something else? Let me know how I can assist you properly!
Failed to parse up to error: [json.exception.parse_error.101] parse error at line 2, column 50: syntax error while parsing value - unexpected end of input; expected '[', '{', or a literal: <<<
{"name": "tool_name", "arguments": {"argument1": >>>
Partial parse: JSON
Parsed message: {"role":"assistant","content":"I apologize for the confusion earlier. Let me clarify how to properly format a tool call.\n\nA valid tool call should look like this:\n\nI apologize for the confusion earlier. Let me clarify how to properly format a tool call.\n\nA valid tool call should look like this:\n\n\n<tool_call>\n{\"name\": \"tool_name\", \"arguments\": {\"argument1\": value1, \"argument2\": value2, ...}}\n</tool_call>\n\n\nFor example, if I wanted to list the contents of your Downloads folder, I would format it like this:\n\n\n<tool_call>\n{\"name\": \"tool_list_dir_post\", \"arguments\": {\"path\": \"/home/rosa/Downloads\"}}\n</tool_call>\n\n\nNow, since you mentioned \"shoopuf\" and I can't find it, could you clarify what you're trying to do? Are you looking for a file, a note, or something else? Let me know how I can assist you properly!"}
[New LWP 927665]
[New LWP 927664]
[New LWP 927663]
[New LWP 927662]
[New LWP 927661]
[New LWP 927660]
[New LWP 927659]
[New LWP 927658]
[New LWP 927657]
[New LWP 927656]
[New LWP 927655]
[New LWP 927654]
[New LWP 927653]
[New LWP 927652]
[New LWP 927633]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
0x00007f2f5989f042 in ?? () from /usr/lib/libc.so.6
#0 0x00007f2f5989f042 in ?? () from /usr/lib/libc.so.6
#1 0x00007f2f598931ac in ?? () from /usr/lib/libc.so.6
#2 0x00007f2f598931f4 in ?? () from /usr/lib/libc.so.6
#3 0x00007f2f59903dcf in wait4 () from /usr/lib/libc.so.6
#4 0x00007f2f5a2bb4f1 in ggml_print_backtrace () from /usr/lib/libggml-base.so
#5 0x00007f2f5a2d077d in ?? () from /usr/lib/libggml-base.so
#6 0x00007f2f59cb1eba in __cxxabiv1::__terminate (handler=<optimized out>) at /usr/src/debug/gcc/gcc/libstdc++-v3/libsupc++/eh_terminate.cc:48
warning: 48 /usr/src/debug/gcc/gcc/libstdc++-v3/libsupc++/eh_terminate.cc: No such file or directory
#7 0x00007f2f59c975d9 in std::terminate () at /usr/src/debug/gcc/gcc/libstdc++-v3/libsupc++/eh_terminate.cc:58
58 in /usr/src/debug/gcc/gcc/libstdc++-v3/libsupc++/eh_terminate.cc
#8 0x00007f2f59cb2176 in __cxxabiv1::__cxa_throw (obj=<optimized out>, tinfo=0x7f2f59e84d90 <typeinfo for std::runtime_error>, dest=0x7f2f59cc9d50 <std::runtime_error::~runtime_error()>) at /usr/src/debug/gcc/gcc/libstdc++-v3/libsupc++/eh_throw.cc:98
warning: 98 /usr/src/debug/gcc/gcc/libstdc++-v3/libsupc++/eh_throw.cc: No such file or directory
#9 0x0000557e58b52487 in ?? ()
#10 0x0000557e58b484af in ?? ()
#11 0x0000557e58b41ec2 in ?? ()
#12 0x0000557e58b24f6d in ?? ()
#13 0x00007f2f59827675 in ?? () from /usr/lib/libc.so.6
#14 0x00007f2f59827729 in __libc_start_main () from /usr/lib/libc.so.6
#15 0x0000557e58adefd5 in ?? ()
[Inferior 1 (process 927625) detached]
terminate called after throwing an instance of 'std::runtime_error'
what(): Invalid diff: now finding less tool calls!
wow i managed to make it happen without even manually crafting a tool call
\s*)?(?:<function_call>|<tools>|<xml><json>|<response>)?\s*\{\s*"name"\s*:\s*"(?:t_get_datetime|t_get_system_info|t_get_network_interfaces|t_get_disk_usage|t_fetch_man_page|t_get_running_system_processes|t_get_running_user_processes|t_get_home_dir_path|t_kill_process|t_lock_screen|t_list_user_services|t_list_system_services|t_system_service_status|t_user_service_status|t_start_user_service|t_restart_user_service|t_stop_user_service|t_kill_user_service|t_media_currently_playing|t_media_toggle_pause|t_media_next|t_media_previous|t_media_toggle_shuffle|t_media_stop|t_list_dir|t_create_dir|t_create_file|t_write_file|t_append_to_file|t_move_file|t_move_multiple_files|t_delete_file|t_empty_trash|t_read_file_or_url|t_read_multiple_files_or_urls|t_ping|t_list_open_ports|t_traceroute|t_whois|t_create_note|t_read_note|t_edit_note|t_delete_note|t_search_notes)"))[\s\S]*`
srv add_waiting_: add task 7330 to waiting list. current waiting = 0 (before add)
que post: new task, id = 7330/1, front = 0
que start_loop: processing new tasks
que start_loop: processing task, id = 7330
slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.636 (> 0.100 thold), f_keep = 0.692
slot reset: id 0 | task -1 |
slot launch_slot_: id 0 | task -1 | launching slot : {"id":0,"n_ctx":16384,"speculative":false,"is_processing":false}
slot launch_slot_: id 0 | task 7330 | processing task
que start_loop: update slots
srv update_slots: posting NEXT_RESPONSE
que post: new task, id = 7331, front = 0
slot update_slots: id 0 | task 7330 | new prompt, n_ctx_slot = 16384, n_keep = 0, n_prompt_tokens = 12365
slot update_slots: id 0 | task 7330 | old: ...
<|im_start|>assistant
| <tool_call>
{"name": "
slot update_slots: id 0 | task 7330 | new: ...
<|im_start|>assistant
| ""{\n
slot update_slots: id 0 | task 7330 | 198 151644 77091 198 151657 198 4913 606 788 330
slot update_slots: id 0 | task 7330 | 198 151644 77091 198 67186 13492 26 35702 77 220
slot update_slots: id 0 | task 7330 | n_past = 7862, memory_seq_rm [7862, end)
slot update_slots: id 0 | task 7330 | prompt processing progress, n_past = 9910, n_tokens = 2048, progress = 0.801456
srv update_slots: decoding batch, n_tokens = 2048
clear_adapter_lora: call
set_embeddings: value = 0
srv update_slots: run slots completed
que start_loop: waiting for new tasks
que start_loop: processing new tasks
que start_loop: processing task, id = 7331
que start_loop: update slots
srv update_slots: posting NEXT_RESPONSE
que post: new task, id = 7332, front = 0
slot update_slots: id 0 | task 7330 | n_past = 9910, memory_seq_rm [9910, end)
slot update_slots: id 0 | task 7330 | prompt processing progress, n_past = 11958, n_tokens = 2048, progress = 0.967085
srv update_slots: decoding batch, n_tokens = 2048
clear_adapter_lora: call
set_embeddings: value = 0
srv update_slots: run slots completed
que start_loop: waiting for new tasks
que start_loop: processing new tasks
que start_loop: processing task, id = 7332
que start_loop: update slots
srv update_slots: posting NEXT_RESPONSE
que post: new task, id = 7333, front = 0
slot update_slots: id 0 | task 7330 | n_past = 11958, memory_seq_rm [11958, end)
slot update_slots: id 0 | task 7330 | prompt processing progress, n_past = 12365, n_tokens = 407, progress = 1.000000
slot update_slots: id 0 | task 7330 | prompt done, n_past = 12365, n_tokens = 407
srv update_slots: decoding batch, n_tokens = 407
clear_adapter_lora: call
set_embeddings: value = 0
Grammar still awaiting trigger after token 1 (`"`)
srv update_chat_: Parsing chat message: "
Parsing input with format Hermes 2 Pro: "
srv send: sending result for task id = 7330
srv send: task id = 7330 pushed to result queue
slot process_toke: id 0 | task 7330 | n_decoded = 1, n_remaining = 989, next token: 1 '"'
srv update_slots: run slots completed
que start_loop: waiting for new tasks
que start_loop: processing new tasks
que start_loop: processing task, id = 7333
que start_loop: update slots
srv update_slots: posting NEXT_RESPONSE
que post: new task, id = 7334, front = 0
slot update_slots: id 0 | task 7330 | slot decode token, n_ctx = 16384, n_past = 12366, n_cache_tokens = 12366, truncated = 0
srv update_slots: decoding batch, n_tokens = 1
clear_adapter_lora: call
set_embeddings: value = 0
data stream, to_send: data: {"choices":[{"finish_reason":null,"index":0,"delta":{"role":"assistant","content":null}}],"created":1761157589,"id":"chatcmpl-x3wtZz2KDXrTFoh936P77CzLwX6oRYbX","model":"/etc/llama-swap/models/Qwen3-30B-A3B-Instruct-2507-Q6_K.gguf","system_fingerprint":"b1032-034e7aa","object":"chat.completion.chunk"}
data stream, to_send: data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":"\""}}],"created":1761157589,"id":"chatcmpl-x3wtZz2KDXrTFoh936P77CzLwX6oRYbX","model":"/etc/llama-swap/models/Qwen3-30B-A3B-Instruct-2507-Q6_K.gguf","system_fingerprint":"b1032-034e7aa","object":"chat.completion.chunk"}
Grammar still awaiting trigger after token 4913 (`{"`)
srv update_chat_: Parsing chat message: "{"
Parsing input with format Hermes 2 Pro: "{"
Partial parse: (?:(```(?:xml|json)?\n\s*)?(<tool_call>|<function_call>|<tool>|<tools>|<response>|<json>|<xml>|<JSON>)?(\s*\{\s*"name"))|<function=([^>]+)>|<function name="([^"]+)">
srv send: sending result for task id = 7330
srv send: task id = 7330 pushed to result queue
slot process_toke: id 0 | task 7330 | n_decoded = 2, n_remaining = 989, next token: 4913 '{"'
srv update_slots: run slots completed
que start_loop: waiting for new tasks
que start_loop: processing new tasks
que start_loop: processing task, id = 7334
que start_loop: update slots
srv update_slots: posting NEXT_RESPONSE
que post: new task, id = 7335, front = 0
slot update_slots: id 0 | task 7330 | slot decode token, n_ctx = 16384, n_past = 12367, n_cache_tokens = 12367, truncated = 0
srv update_slots: decoding batch, n_tokens = 1
clear_adapter_lora: call
set_embeddings: value = 0
Grammar still awaiting trigger after token 606 (`name`)
srv update_chat_: Parsing chat message: "{"name
Parsing input with format Hermes 2 Pro: "{"name
Partial parse: (?:(```(?:xml|json)?\n\s*)?(<tool_call>|<function_call>|<tool>|<tools>|<response>|<json>|<xml>|<JSON>)?(\s*\{\s*"name"))|<function=([^>]+)>|<function name="([^"]+)">
srv send: sending result for task id = 7330
srv send: task id = 7330 pushed to result queue
slot process_toke: id 0 | task 7330 | n_decoded = 3, n_remaining = 989, next token: 606 'name'
srv update_slots: run slots completed
que start_loop: waiting for new tasks
que start_loop: processing new tasks
que start_loop: processing task, id = 7335
que start_loop: update slots
srv update_slots: posting NEXT_RESPONSE
que post: new task, id = 7336, front = 0
slot update_slots: id 0 | task 7330 | slot decode token, n_ctx = 16384, n_past = 12368, n_cache_tokens = 12368, truncated = 0
srv update_slots: decoding batch, n_tokens = 1
clear_adapter_lora: call
set_embeddings: value = 0
Grammar still awaiting trigger after token 788 (`":`)
srv update_chat_: Parsing chat message: "{"name":
Parsing input with format Hermes 2 Pro: "{"name":
Failed to parse up to error: [json.exception.parse_error.101] parse error at line 1, column 9: syntax error while parsing value - unexpected end of input; expected '[', '{', or a literal: <<<{"name":>>>
Parsed partial JSON: {"name":"1922538714"} (json_healing_marker: "1922538714)
Cleaned up JSON {"name":"1922538714"} to {} (json_healing_marker : '"1922538714')
Partial parse: incomplete tool call
srv send: sending result for task id = 7330
srv send: task id = 7330 pushed to result queue
slot process_toke: id 0 | task 7330 | n_decoded = 4, n_remaining = 989, next token: 788 '":'
srv update_slots: run slots completed
que start_loop: waiting for new tasks
que start_loop: processing new tasks
que start_loop: processing task, id = 7336
que start_loop: update slots
srv update_slots: posting NEXT_RESPONSE
que post: new task, id = 7337, front = 0
slot update_slots: id 0 | task 7330 | slot decode token, n_ctx = 16384, n_past = 12369, n_cache_tokens = 12369, truncated = 0
srv update_slots: decoding batch, n_tokens = 1
clear_adapter_lora: call
set_embeddings: value = 0
Grammar still awaiting trigger after token 330 (` "`)
srv update_chat_: Parsing chat message: "{"name": "
Parsing input with format Hermes 2 Pro: "{"name": "
Failed to parse up to error: [json.exception.parse_error.101] parse error at line 1, column 11: syntax error while parsing value - invalid string: missing closing quote; last read: '"': <<<{"name": ">>>
Parsed partial JSON: {"name":"1383794960"} (json_healing_marker: 1383794960)
Cleaned up JSON {"name":"1383794960"} to {} (json_healing_marker : '1383794960')
Partial parse: incomplete tool call
srv send: sending result for task id = 7330
srv send: task id = 7330 pushed to result queue
slot process_toke: id 0 | task 7330 | n_decoded = 5, n_remaining = 989, next token: 330 ' "'
srv update_slots: run slots completed
que start_loop: waiting for new tasks
que start_loop: processing new tasks
que start_loop: processing task, id = 7337
que start_loop: update slots
srv update_slots: posting NEXT_RESPONSE
que post: new task, id = 7338, front = 0
slot update_slots: id 0 | task 7330 | slot decode token, n_ctx = 16384, n_past = 12370, n_cache_tokens = 12370, truncated = 0
srv update_slots: decoding batch, n_tokens = 1
clear_adapter_lora: call
set_embeddings: value = 0
Grammar still awaiting trigger after token 83 (`t`)
srv update_chat_: Parsing chat message: "{"name": "t
Parsing input with format Hermes 2 Pro: "{"name": "t
Failed to parse up to error: [json.exception.parse_error.101] parse error at line 1, column 12: syntax error while parsing value - invalid string: missing closing quote; last read: '"t': <<<{"name": "t>>>
Parsed partial JSON: {"name":"t1540478823"} (json_healing_marker: 1540478823)
Cleaned up JSON {"name":"t1540478823"} to {} (json_healing_marker : '1540478823')
Partial parse: incomplete tool call
srv send: sending result for task id = 7330
srv send: task id = 7330 pushed to result queue
slot process_toke: id 0 | task 7330 | n_decoded = 6, n_remaining = 989, next token: 83 't'
srv update_slots: run slots completed
que start_loop: waiting for new tasks
que start_loop: processing new tasks
que start_loop: processing task, id = 7338
que start_loop: update slots
srv update_slots: posting NEXT_RESPONSE
que post: new task, id = 7339, front = 0
slot update_slots: id 0 | task 7330 | slot decode token, n_ctx = 16384, n_past = 12371, n_cache_tokens = 12371, truncated = 0
srv update_slots: decoding batch, n_tokens = 1
clear_adapter_lora: call
set_embeddings: value = 0
Grammar still awaiting trigger after token 29173 (`_media`)
srv update_chat_: Parsing chat message: "{"name": "t_media
Parsing input with format Hermes 2 Pro: "{"name": "t_media
Failed to parse up to error: [json.exception.parse_error.101] parse error at line 1, column 18: syntax error while parsing value - invalid string: missing closing quote; last read: '"t_media': <<<{"name": "t_media>>>
Parsed partial JSON: {"name":"t_media287057570"} (json_healing_marker: 287057570)
Cleaned up JSON {"name":"t_media287057570"} to {} (json_healing_marker : '287057570')
Partial parse: incomplete tool call
srv send: sending result for task id = 7330
srv send: task id = 7330 pushed to result queue
slot process_toke: id 0 | task 7330 | n_decoded = 7, n_remaining = 989, next token: 29173 '_media'
srv update_slots: run slots completed
que start_loop: waiting for new tasks
que start_loop: processing new tasks
que start_loop: processing task, id = 7339
que start_loop: update slots
srv update_slots: posting NEXT_RESPONSE
que post: new task, id = 7340, front = 0
slot update_slots: id 0 | task 7330 | slot decode token, n_ctx = 16384, n_past = 12372, n_cache_tokens = 12372, truncated = 0
srv update_slots: decoding batch, n_tokens = 1
clear_adapter_lora: call
set_embeddings: value = 0
Grammar still awaiting trigger after token 48860 (`_toggle`)
srv update_chat_: Parsing chat message: "{"name": "t_media_toggle
Parsing input with format Hermes 2 Pro: "{"name": "t_media_toggle
Failed to parse up to error: [json.exception.parse_error.101] parse error at line 1, column 25: syntax error while parsing value - invalid string: missing closing quote; last read: '"t_media_toggle': <<<{"name": "t_media_toggle>>>
Parsed partial JSON: {"name":"t_media_toggle1129250011"} (json_healing_marker: 1129250011)
Cleaned up JSON {"name":"t_media_toggle1129250011"} to {} (json_healing_marker : '1129250011')
Partial parse: incomplete tool call
srv send: sending result for task id = 7330
srv send: task id = 7330 pushed to result queue
slot process_toke: id 0 | task 7330 | n_decoded = 8, n_remaining = 989, next token: 48860 '_toggle'
srv update_slots: run slots completed
que start_loop: waiting for new tasks
que start_loop: processing new tasks
que start_loop: processing task, id = 7340
que start_loop: update slots
srv update_slots: posting NEXT_RESPONSE
que post: new task, id = 7341, front = 0
slot update_slots: id 0 | task 7330 | slot decode token, n_ctx = 16384, n_past = 12373, n_cache_tokens = 12373, truncated = 0
srv update_slots: decoding batch, n_tokens = 1
clear_adapter_lora: call
set_embeddings: value = 0
Grammar still awaiting trigger after token 59989 (`_pause`)
srv update_chat_: Parsing chat message: "{"name": "t_media_toggle_pause
Parsing input with format Hermes 2 Pro: "{"name": "t_media_toggle_pause
Failed to parse up to error: [json.exception.parse_error.101] parse error at line 1, column 31: syntax error while parsing value - invalid string: missing closing quote; last read: '"t_media_toggle_pause': <<<{"name": "t_media_toggle_pause>>>
Parsed partial JSON: {"name":"t_media_toggle_pause578173865"} (json_healing_marker: 578173865)
Cleaned up JSON {"name":"t_media_toggle_pause578173865"} to {} (json_healing_marker : '578173865')
Partial parse: incomplete tool call
srv send: sending result for task id = 7330
srv send: task id = 7330 pushed to result queue
slot process_toke: id 0 | task 7330 | n_decoded = 9, n_remaining = 989, next token: 59989 '_pause'
srv update_slots: run slots completed
que start_loop: waiting for new tasks
que start_loop: processing new tasks
que start_loop: processing task, id = 7341
que start_loop: update slots
srv update_slots: posting NEXT_RESPONSE
que post: new task, id = 7342, front = 0
slot update_slots: id 0 | task 7330 | slot decode token, n_ctx = 16384, n_past = 12374, n_cache_tokens = 12374, truncated = 0
srv update_slots: decoding batch, n_tokens = 1
clear_adapter_lora: call
set_embeddings: value = 0
Grammar still awaiting trigger after token 497 (`",`)
srv update_chat_: Parsing chat message: "{"name": "t_media_toggle_pause",
Parsing input with format Hermes 2 Pro: "{"name": "t_media_toggle_pause",
Failed to parse up to error: [json.exception.parse_error.101] parse error at line 1, column 33: syntax error while parsing object key - unexpected end of input; expected string literal: <<<{"name": "t_media_toggle_pause",>>>
Parsed partial JSON: {"name":"t_media_toggle_pause","1471595956":1} (json_healing_marker: "1471595956)
Cleaned up JSON {"name":"t_media_toggle_pause","1471595956":1} to {"name":"t_media_toggle_pause"} (json_healing_marker : '"1471595956')
Partial parse: incomplete tool call
srv send: sending result for task id = 7330
srv send: task id = 7330 pushed to result queue
data stream, to_send: data: {"choices":[{"finish_reason":null,"index":0,"delta":{"tool_calls":[{"index":0,"id":"81kd5emRGvXxWg9kaeMiD63xdkEfvKpy","type":"function","function":{"name":"t_media_toggle_pause","arguments":""}}]}}],"created":1761157589,"id":"chatcmpl-x3wtZz2KDXrTFoh936P77CzLwX6oRYbX","model":"/etc/llama-swap/models/Qwen3-30B-A3B-Instruct-2507-Q6_K.gguf","system_fingerprint":"b1032-034e7aa","object":"chat.completion.chunk"}
slot process_toke: id 0 | task 7330 | n_decoded = 10, n_remaining = 989, next token: 497 '",'
srv update_slots: run slots completed
que start_loop: waiting for new tasks
que start_loop: processing new tasks
que start_loop: processing task, id = 7342
que start_loop: update slots
srv update_slots: posting NEXT_RESPONSE
que post: new task, id = 7343, front = 0
slot update_slots: id 0 | task 7330 | slot decode token, n_ctx = 16384, n_past = 12375, n_cache_tokens = 12375, truncated = 0
srv update_slots: decoding batch, n_tokens = 1
clear_adapter_lora: call
set_embeddings: value = 0
Grammar still awaiting trigger after token 330 (` "`)
srv update_chat_: Parsing chat message: "{"name": "t_media_toggle_pause", "
Parsing input with format Hermes 2 Pro: "{"name": "t_media_toggle_pause", "
Failed to parse up to error: [json.exception.parse_error.101] parse error at line 1, column 35: syntax error while parsing object key - invalid string: missing closing quote; last read: '"'; expected string literal: <<<{"name": "t_media_toggle_pause", ">>>
Parsed partial JSON: {"name":"t_media_toggle_pause","862248064":1} (json_healing_marker: 862248064)
Cleaned up JSON {"name":"t_media_toggle_pause","862248064":1} to {"name":"t_media_toggle_pause"} (json_healing_marker : '862248064')
Partial parse: incomplete tool call
srv send: sending result for task id = 7330
srv send: task id = 7330 pushed to result queue
slot process_toke: id 0 | task 7330 | n_decoded = 11, n_remaining = 989, next token: 330 ' "'
srv update_slots: run slots completed
que start_loop: waiting for new tasks
que start_loop: processing new tasks
que start_loop: processing task, id = 7343
que start_loop: update slots
srv update_slots: posting NEXT_RESPONSE
que post: new task, id = 7344, front = 0
slot update_slots: id 0 | task 7330 | slot decode token, n_ctx = 16384, n_past = 12376, n_cache_tokens = 12376, truncated = 0
srv update_slots: decoding batch, n_tokens = 1
clear_adapter_lora: call
set_embeddings: value = 0
Grammar still awaiting trigger after token 16370 (`arguments`)
srv update_chat_: Parsing chat message: "{"name": "t_media_toggle_pause", "arguments
Parsing input with format Hermes 2 Pro: "{"name": "t_media_toggle_pause", "arguments
Failed to parse up to error: [json.exception.parse_error.101] parse error at line 1, column 44: syntax error while parsing object key - invalid string: missing closing quote; last read: '"arguments'; expected string literal: <<<{"name": "t_media_toggle_pause", "arguments>>>
Parsed partial JSON: {"name":"t_media_toggle_pause","arguments1759466821":1} (json_healing_marker: 1759466821)
Cleaned up JSON {"name":"t_media_toggle_pause","arguments1759466821":1} to {"name":"t_media_toggle_pause"} (json_healing_marker : '1759466821')
Partial parse: incomplete tool call
srv send: sending result for task id = 7330
srv send: task id = 7330 pushed to result queue
slot process_toke: id 0 | task 7330 | n_decoded = 12, n_remaining = 989, next token: 16370 'arguments'
srv update_slots: run slots completed
que start_loop: waiting for new tasks
que start_loop: processing new tasks
que start_loop: processing task, id = 7344
que start_loop: update slots
srv update_slots: posting NEXT_RESPONSE
que post: new task, id = 7345, front = 0
slot update_slots: id 0 | task 7330 | slot decode token, n_ctx = 16384, n_past = 12377, n_cache_tokens = 12377, truncated = 0
srv update_slots: decoding batch, n_tokens = 1
clear_adapter_lora: call
set_embeddings: value = 0
Grammar still awaiting trigger after token 788 (`":`)
srv update_chat_: Parsing chat message: "{"name": "t_media_toggle_pause", "arguments":
Parsing input with format Hermes 2 Pro: "{"name": "t_media_toggle_pause", "arguments":
Failed to parse up to error: [json.exception.parse_error.101] parse error at line 1, column 46: syntax error while parsing value - unexpected end of input; expected '[', '{', or a literal: <<<{"name": "t_media_toggle_pause", "arguments":>>>
Parsed partial JSON: {"name":"t_media_toggle_pause","arguments":"1261084216"} (json_healing_marker: "1261084216)
Cleaned up JSON {"name":"t_media_toggle_pause","arguments":"1261084216"} to {"name":"t_media_toggle_pause"} (json_healing_marker : '"1261084216')
Partial parse: incomplete tool call
srv send: sending result for task id = 7330
srv send: task id = 7330 pushed to result queue
slot process_toke: id 0 | task 7330 | n_decoded = 13, n_remaining = 989, next token: 788 '":'
srv update_slots: run slots completed
que start_loop: waiting for new tasks
que start_loop: processing new tasks
que start_loop: processing task, id = 7345
que start_loop: update slots
srv update_slots: posting NEXT_RESPONSE
que post: new task, id = 7346, front = 0
slot update_slots: id 0 | task 7330 | slot decode token, n_ctx = 16384, n_past = 12378, n_cache_tokens = 12378, truncated = 0
srv update_slots: decoding batch, n_tokens = 1
clear_adapter_lora: call
set_embeddings: value = 0
Grammar still awaiting trigger after token 314 (` {`)
srv update_chat_: Parsing chat message: "{"name": "t_media_toggle_pause", "arguments": {
Parsing input with format Hermes 2 Pro: "{"name": "t_media_toggle_pause", "arguments": {
Failed to parse up to error: [json.exception.parse_error.101] parse error at line 1, column 48: syntax error while parsing object key - unexpected end of input; expected string literal: <<<{"name": "t_media_toggle_pause", "arguments": {>>>
Parsed partial JSON: {"name":"t_media_toggle_pause","arguments":{"105798202":1}} (json_healing_marker: "105798202)
Cleaned up JSON {"name":"t_media_toggle_pause","arguments":{"105798202":1}} to {"name":"t_media_toggle_pause","arguments":"{"} (json_healing_marker : '"105798202')
Partial parse: incomplete tool call
srv send: sending result for task id = 7330
srv send: task id = 7330 pushed to result queue
slot process_toke: id 0 | task 7330 | n_decoded = 14, n_remaining = 989, next token: 314 ' {'
srv update_slots: run slots completed
que start_loop: waiting for new tasks
que start_loop: processing new tasks
que start_loop: processing task, id = 7346
que start_loop: update slots
srv update_slots: posting NEXT_RESPONSE
que post: new task, id = 7347, front = 0
slot update_slots: id 0 | task 7330 | slot decode token, n_ctx = 16384, n_past = 12379, n_cache_tokens = 12379, truncated = 0
srv update_slots: decoding batch, n_tokens = 1
clear_adapter_lora: call
set_embeddings: value = 0
data stream, to_send: data: {"choices":[{"finish_reason":null,"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"{"}}]}}],"created":1761157589,"id":"chatcmpl-x3wtZz2KDXrTFoh936P77CzLwX6oRYbX","model":"/etc/llama-swap/models/Qwen3-30B-A3B-Instruct-2507-Q6_K.gguf","system_fingerprint":"b1032-034e7aa","object":"chat.completion.chunk"}
Grammar still awaiting trigger after token 3417 (`}}`)
srv update_chat_: Parsing chat message: "{"name": "t_media_toggle_pause", "arguments": {}}
Parsing input with format Hermes 2 Pro: "{"name": "t_media_toggle_pause", "arguments": {}}
Parsed partial JSON: {"name":"t_media_toggle_pause","arguments":{}} (json_healing_marker: )
Cleaned up JSON {"name":"t_media_toggle_pause","arguments":{}} to {"name":"t_media_toggle_pause","arguments":"{}"} (json_healing_marker : '')
srv send: sending result for task id = 7330
srv send: task id = 7330 pushed to result queue
slot process_toke: id 0 | task 7330 | n_decoded = 15, n_remaining = 989, next token: 3417 '}}'
srv update_slots: run slots completed
que start_loop: waiting for new tasks
que start_loop: processing new tasks
que start_loop: processing task, id = 7347
que start_loop: update slots
srv update_slots: posting NEXT_RESPONSE
que post: new task, id = 7348, front = 0
slot update_slots: id 0 | task 7330 | slot decode token, n_ctx = 16384, n_past = 12380, n_cache_tokens = 12380, truncated = 0
srv update_slots: decoding batch, n_tokens = 1
clear_adapter_lora: call
set_embeddings: value = 0
data stream, to_send: data: {"choices":[{"finish_reason":null,"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"}"}}]}}],"created":1761157589,"id":"chatcmpl-x3wtZz2KDXrTFoh936P77CzLwX6oRYbX","model":"/etc/llama-swap/models/Qwen3-30B-A3B-Instruct-2507-Q6_K.gguf","system_fingerprint":"b1032-034e7aa","object":"chat.completion.chunk"}
Grammar still awaiting trigger after token 1 (`"`)
srv update_chat_: Parsing chat message: "{"name": "t_media_toggle_pause", "arguments": {}}"
Parsing input with format Hermes 2 Pro: "{"name": "t_media_toggle_pause", "arguments": {}}"
Failed to parse up to error: [json.exception.parse_error.101] parse error at line 1, column 51: syntax error while parsing value - invalid string: missing closing quote; last read: '"'; expected end of input: <<<{"name": "t_media_toggle_pause", "arguments": {}}">>>
Partial parse: failed to parse tool call
[New LWP 279355]
[New LWP 279354]
[New LWP 279353]
[New LWP 279352]
[New LWP 279351]
[New LWP 279339]
[New LWP 279258]
[New LWP 279257]
[New LWP 279256]
[New LWP 279255]
[New LWP 279254]
[New LWP 279253]
[New LWP 279252]
[New LWP 279251]
[New LWP 279250]
[New LWP 279249]
[New LWP 279248]
[New LWP 279247]
[New LWP 279246]
[New LWP 279212]
This GDB supports auto-downloading debuginfo from the following URLs:
<https://debuginfod.archlinux.org>
Enable debuginfod for this session? (y or [n]) [answered N; input not from terminal]
Debuginfod has been disabled.
To make this setting permanent, add 'set debuginfod enabled off' to .gdbinit.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
0x00007fbe8189f042 in ?? () from /usr/lib/libc.so.6
#0 0x00007fbe8189f042 in ?? () from /usr/lib/libc.so.6
#1 0x00007fbe818931ac in ?? () from /usr/lib/libc.so.6
#2 0x00007fbe818931f4 in ?? () from /usr/lib/libc.so.6
#3 0x00007fbe81903dcf in wait4 () from /usr/lib/libc.so.6
#4 0x00007fbe81f7a4f1 in ggml_print_backtrace () from /usr/lib/libggml-base.so
#5 0x00007fbe81f8f77d in ?? () from /usr/lib/libggml-base.so
#6 0x00007fbe81cb1eba in __cxxabiv1::__terminate (handler=<optimized out>) at /usr/src/debug/gcc/gcc/libstdc++-v3/libsupc++/eh_terminate.cc:48
warning: 48 /usr/src/debug/gcc/gcc/libstdc++-v3/libsupc++/eh_terminate.cc: No such file or directory
#7 0x00007fbe81c975d9 in std::terminate () at /usr/src/debug/gcc/gcc/libstdc++-v3/libsupc++/eh_terminate.cc:58
58 in /usr/src/debug/gcc/gcc/libstdc++-v3/libsupc++/eh_terminate.cc
#8 0x00007fbe81cb2176 in __cxxabiv1::__cxa_throw (obj=<optimized out>, tinfo=0x7fbe81e84d90 <typeinfo for std::runtime_error>, dest=0x7fbe81cc9d50 <std::runtime_error::~runtime_error()>) at /usr/src/debug/gcc/gcc/libstdc++-v3/libsupc++/eh_throw.cc:98
warning: 98 /usr/src/debug/gcc/gcc/libstdc++-v3/libsupc++/eh_throw.cc: No such file or directory
#9 0x0000564367a61487 in ?? ()
#10 0x0000564367a58a49 in ?? ()
#11 0x0000564367a5a1b8 in ?? ()
#12 0x0000564367a50ea4 in ?? ()
#13 0x0000564367a33f6d in ?? ()
#14 0x00007fbe81827675 in ?? () from /usr/lib/libc.so.6
#15 0x00007fbe81827729 in __libc_start_main () from /usr/lib/libc.so.6
#16 0x00005643679edfd5 in ?? ()
[Inferior 1 (process 279211) detached]
terminate called after throwing an instance of 'std::runtime_error'
what(): Invalid diff: now finding less tool calls!
ai/scripts/start_llama.sh: line 35: 279211 Aborted (core dumped) /usr/bin/llama-server --host 0.0.0.0 --port 12434 -sm none -mg 0 --flash-attn on --kv-unified --jinja -ngl 999 --n-cpu-moe 25 --verbose $SAMPLING_PARAMS -m $MODEL_PATH/$AI_MODEL
