Add RimeNonJsonTTSService for non-JSON WebSocket API support
Codecov Report
:x: Patch coverage is 0% with 158 lines in your changes missing coverage. Please review.
| Files with missing lines | Patch % | Lines |
|---|---|---|
| src/pipecat/services/rime/tts.py | 0.00% | 158 Missing :warning: |
| Files with missing lines | Coverage Δ | |
|---|---|---|
| src/pipecat/services/rime/tts.py | 0.00% <0.00%> (ø) |
... and 8 files with indirect coverage changes
:rocket: New features to boost your workflow:
- :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
#3063
@markbackman, any update here?
@markbackman let me test this, and I will get back to you on this.
@markbackman One of the key observations is that it works correctly in InterruptibleTTSService but not in WebsocketTTSService. I suspect this points to an issue in our clear-message mechanism. In InterruptibleTTSService the client disconnects and reconnects, which is likely why we are not seeing the problem there.
@markbackman One of the key observations is that it works correctly in
InterruptibleTTSServicebut not inWebsocketTTSService. I suspect this points to an issue in our clear-message mechanism. InInterruptibleTTSServicethe client disconnects and reconnects, which is likely why we are not seeing the problem there.
The InterruptibleTTSService has a _handle_interruption() method:
async def _handle_interruption(self, frame: InterruptionFrame, direction: FrameDirection):
await super()._handle_interruption(frame, direction)
if self._bot_speaking:
await self._disconnect()
await self._connect()
This disconnects and reconnects the websocket. Ideally, this is avoided because there are scenarios where the websocket teardown and reconnection take time.
Ideally, we would use the CLEAR message to clear the current generation requests as it's more precise. If that's not possible, then we should remove the sending of the CLEAR message and instead rely on the InterruptibleTTSService base class.
@markbackman After investigating with the backend team, it turns out the CLEAR message doesn’t actually interrupt the audio, which explains why audio continues after CLEAR. The non JSON path wasn’t designed to perform a full cancellation. Given that, I think the next step is to rely on InterruptibleTTSService for real interruption and remove our _handle_interruption logic.