`UpdatePrompt` messages require multiple attempts before taking effect despite `PromptUpdated`

[camachomarco]

What is the current behavior?

When sending UpdatePrompt messages via the Voice Agent WebSocket API from within the ConversationText handler after catching the user's message, the prompt updates are acknowledged with PromptUpdated responses but do not take immediate effect. The agent continues to use the original prompt/persona for several conversation turns, requiring multiple successive UpdatePrompt calls before the new prompt actually overrides the agent's behavior.

In our testing, it took 4 consecutive UpdatePrompt messages over ~2 minutes before the agent finally adopted the new prompt, despite receiving PromptUpdated confirmations for after each attempt.

Steps to reproduce

1. Establish a Voice Agent WebSocket connection with initial settings containing a specific prompt (e.g., "You are a helpful assistant named Lua")
2. Send an UpdatePrompt message with a drastically different prompt (e.g., a rude persona named Alix)
3. Observe that the agent acknowledges with PromptUpdated but continues using the original prompt
4. Send the same UpdatePrompt message again with each user interaction
5. Eventually (after 3-4 attempts), the agent will suddenly adopt the new prompt

Code example:

    async def _on_conversation_text(self, client, conversation_text=None, **kwargs) -> None:
        """Forward transcription to client"""
        if conversation_text is None:
            conversation_text = {}

        role = getattr(conversation_text, "role", "unknown")
        content = getattr(conversation_text, "content", "")

        # Try to update prompt with every user message
        if role == "user" and content.strip():
            logger.info(f"User message: '{content}' - sending prompt update")
            
            rude_prompt = """You are NOT a helpful assistant. Your name is Alix. You are an extremely rude and sarcastic AI assistant."""
            
            try:
                # Create and send UpdatePrompt message
                update_prompt_message = UpdatePromptOptions()
                update_prompt_message.prompt = rude_prompt
                
                message_json = json.dumps(update_prompt_message.to_dict())
                await self.agent_connection.send(message_json)
                
            except Exception as e:
                logger.error(f"Error sending UpdatePrompt: {str(e)}")

Expected behavior

When an UpdatePrompt message is sent and acknowledged with PromptUpdated, the agent should immediately use the new prompt for subsequent responses. The prompt update should take effect on the current agent respone or, at least, very next agent response, not after multiple attempts.

Please tell us about your environment

• Operating System/Version: Windows 11
• Client that interacts with Deepgram's server: FastAPI app in Docker container in Ubuntu (WSL2)
• Python Version: 3.12.4
• Deepgram SDK Version: deepgram-sdk v4.1.0

Other information

Observed behavior pattern:

• 1st UpdatePrompt: Acknowledged but ignored
• 2nd UpdatePrompt: Acknowledged but ignored
• 3rd UpdatePrompt: Acknowledged but ignored
• 4th UpdatePrompt: Finally takes effect

Logs showing the issue:

INFO - Sending UpdatePrompt with rude prompt
WARNING - Unknown Message: response_type: PromptUpdated, data: {'type': 'PromptUpdated'}
# Agent still responds as "Lua" instead of adopting rude persona

Potential root causes:

1. Context precedence: Original prompt has stronger weight/priority
2. Buffering/caching: Updates are queued but not applied immediately
3. Race conditions: Updates sent during active conversation turns (Before the Voice Agent responds or even thinks) get delayed
4. Accumulative behavior: Multiple updates stack up before overriding original context

Workaround considerations:

• Sending updates only at connection start (not mid-conversation) might be more reliable
• Sending updates only after the VA has responded might be more reliable (I tested this and it works fine)
• The current behavior makes dynamic prompt updates unreliable for real-time applications

This behavior suggests that UpdatePrompt functionality may not be working as designed, or there's insufficient documentation about timing/context requirements for prompt updates to take effect.