pipecat-ai
Interruptions not working with Twilio
I ran the sample code for Twilio examples/twilio-chatbot but noticed that the user cannot interrupt the AI. I even set the handle_sigint in the PipelineRunner parameter to False but it's still not working. My hunch is that the voice output is already sent to Twilio and therefore cannot be interrupted.
I am facing the same issue. Any luck?
I tried pushing different frames like CancelFrame etc when StartInterruptionFrame frame is broadcasted but no luck. How to fix it?
I try to use "clear" message to stop the twilio output audio and it work
if isinstance(frame, StartInterruptionFrame):
answer = {"event": "clear", "streamSid": self.stream_sid}
await self._websocket.send_text(json.dumps(answer))
ref: https://www.twilio.com/docs/voice/media-streams/websocket-messages#send-a-clear-message
answer = {"event": "clear", "streamSid": self.stream_sid} await self._websocket.send_text(json.dumps(answer))
@nulyang thank you so much. It worked for me as well like a breeze. Appreciate your support!
@nulyang Thanks a lot! This worked for me as well. Might be worth adding this to the library.
@nulyang @nisalr Where should the clear websocket call be made in the twilio example? Or is this fix applied inside the internal code of the package?
Hey @xypnox I added a interruption handler between the stt and tma_in
class InterruptionHandler(FrameProcessor):
def __init__(self, websocket_client: WebSocket, stream_sid: str, call_id: str):
super().__init__()
self.websocket_client = websocket_client
self.stream_sid = stream_sid
self.call_id = call_id
self.last_text_frame_time = time.time()
self.time_since_last_text_frame = 0
async def process_frame(self, frame: Frame, direction: FrameDirection):
if isinstance(frame, StartInterruptionFrame):
buffer_clear_message = {"event": "clear", "streamSid": self.stream_sid}
await self.websocket_client.send_text(json.dumps(buffer_clear_message))
await self.push_frame(frame, direction)
The interruption handler is instantiated
interruption_handler = InterruptionHandler(websocket_client, stream_sid, call_id)
and the Pipeline is defined as
pipeline = Pipeline([
transport.input(), # Websocket input from client
stt, # Speech-To-Text
interruption_handler, # Interruption handler
tma_in, # User responses
llm, # LLM
tts, # Text-To-Speech
transport.output(), # Websocket output to client
tma_out # LLM responses
])
