agents icon indicating copy to clipboard operation
agents copied to clipboard

Tool Call Results Lost During User Interruption Leading to Duplicate Executions

Open jiahao6635 opened this issue 1 month ago β€’ 4 comments

## πŸ› Bug Report: Tool Call Results Lost During User Interruption

### Summary
When a user interrupts the agent during or immediately after tool execution, the completed tool calls and their results are not saved to the chat history. This causes the LLM to be unaware that tools have already been executed in the next inference turn, leading to duplicate tool executions.

### Environment
- **livekit-agents version**: 1.1.6 (verified the issue still exists in 1.2.15)
- **Python version**: 3.9+
- **Use case**: Voice agent handling restaurant reservations

### Impact
- **Critical**: In production environments, this causes duplicate operations (e.g., creating duplicate orders)
- **Data integrity**: Multiple identical records created in database
- **User experience**: Users receive incorrect confirmations and duplicated resources

---

### πŸ”„ Steps to Reproduce

1. **Setup**: Voice agent with a tool that creates resources (e.g., `create_reservation`)
2. **User request**: User initiates a modification flow (e.g., "Change to 6 people")
3. **LLM executes tool**: Agent calls `create_reservation` and successfully creates Order #1
4. **User interrupts**: User speaks (VAD triggered) immediately after tool completion but before agent finishes speaking
5. **LLM continues**: Agent processes user's next input
6. **Result**: Agent calls `create_reservation` again, creating duplicate Order #2

### Expected Behavior
The completed tool call and its result should be saved to chat history even when interrupted, so the LLM knows not to re-execute the same tool.

### Actual Behavior
The tool executes successfully but its result is not recorded in the chat history when interruption occurs, causing the LLM to repeat the tool call.

---

### πŸ“Š Real-World Example

#### Timeline from Production Logs

| Time | Event | Chat History State |
|------|-------|-------------------|
| 14:29:38.705 | User: "ζˆ‘θ¦ζ”Ήζˆ6δΈͺδΊΊ" (Change to 6 people) | - |
| 14:29:50.289 | **LLM outputs tool call**: `create_reservation` | - |
| 14:29:50.583 | **βœ… Order created successfully**: NT02011172025102200047 | ❌ Not recorded |
| 14:29:52.063 | **⚠️ User interrupts** (VAD triggered) | Only partial text saved |
| 14:29:53.736 | **LLM inference #2** starts | ❌ Unaware of Order #047 |
| 14:29:54.599 | LLM outputs tool call again: `create_reservation` | - |
| 14:29:59.909 | **πŸ’₯ Duplicate order created**: NT02011172025102200048 | - |

#### Key Log Excerpts

livekit.agents|14:29:50,584|toolθ€—ζ—Ά:create_reservation elapsed: 292.86 ms livekit.agents|14:29:52,063|Speech handle 打断 livekit.agents|14:29:52,063|_pipeline_reply_task interrupted position2


**Result**: Two identical orders created with same parameters (6 people, 2025-10-22 15:30:00, same location and decorations)

---

### πŸ” Root Cause Analysis

#### Problem Location

**File**: `livekit-agents/livekit/agents/voice/agent_activity.py`
**Function**: `_pipeline_reply_task()`
**Lines**: ~1577-1593 (v1.1.6) and similar in v1.2.15

#### Current Code (Interrupted Path)

```python
if forwarded_text:
    msg = chat_ctx.add_message(
        role="assistant",
        content=forwarded_text,  # ❌ Only saves partial text
        id=llm_gen_data.id,
        interrupted=True,
        created_at=reply_started_at,
    )
    self._agent._chat_ctx.insert(msg)
    self._session._conversation_item_added(msg)

await utils.aio.cancel_and_wait(exe_task)
return  # ❌ Early return skips tool result saving

Issues:

  1. ❌ No tool_calls parameter in the message
  2. ❌ No tool result messages (role="tool") created
  3. ❌ Early return bypasses the tool saving logic at line ~1607

Why Tools Complete But Results Are Lost

File: livekit-agents/livekit/agents/voice/generation.py Function: _execute_tools_task()

except asyncio.CancelledError:
    # Wait for pending tools to complete
    await asyncio.gather(*tasks)  # βœ… Tools do finish
    # ❌ But results in tool_output.output are never accessed

Tools are protected by asyncio.shield() and complete successfully, but when exe_task is cancelled via cancel_and_wait(), the caller never retrieves tool_output.output.

Flow Diagram

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚  LLM generates response + calls tools        β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
               β”‚
        User interrupts?
        β”Œβ”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”
     Yesβ”‚             β”‚No
        ↓             ↓
  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
  β”‚ INT path  β”‚  β”‚ Normal path  β”‚
  β”‚ β€’ Save    β”‚  β”‚ β€’ Save text  β”‚
  β”‚   text    β”‚  β”‚ β€’ βœ… Save    β”‚
  β”‚ β€’ ❌ Skip β”‚  β”‚   tool calls β”‚
  β”‚   tools   β”‚  β”‚ β€’ βœ… Save    β”‚
  β”‚ β€’ return  β”‚  β”‚   results    β”‚
  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

I'm willing to contribute this fix!

jiahao6635 avatar Oct 22 '25 11:10 jiahao6635