faster-whisper-server icon indicating copy to clipboard operation
faster-whisper-server copied to clipboard
verified publisher icon fedirz

Live transcription websocket disconnecting

Open bjornbracko opened this issue 1 year ago • 4 comments

I've been trying to get the live transcription to work, but every time i try i get some websocket disconnect. I also cant get it to work on the demo site.

This is the traceback:

ERROR: Exception in ASGI application

  • Exception Group Traceback (most recent call last): | File "/root/faster-whisper-server/.venv/lib/python3.12/site-packages/uvicorn/protocols/websockets/websockets_impl.py", line 244, in run_asgi | result = await self.app(self.scope, self.asgi_receive, self.asgi_send) # type: ignore[func-returns-value] | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/root/faster-whisper-server/.venv/lib/python3.12/site-packages/uvicorn/middleware/proxy_headers.py", line 70, in call | return await self.app(scope, receive, send) | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | File "/root/faster-whisper-server/.venv/lib/python3.12/site-packages/fastapi/applications.py", line 1054, in call | await super().call(scope, receive, send) | File "/root/faster-whisper-server/.venv/lib/python3.12/site-packages/starlette/applications.py", line 113, in call | await self.middleware_stack(scope, receive, send) | File "/root/faster-whisper-server/.venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 152, in call | await self.app(scope, receive, send) | File "/root/faster-whisper-server/.venv/lib/python3.12/site-packages/starlette/middleware/exceptions.py", line 62, in call | await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send) | File "/root/faster-whisper-server/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 62, in wrapped_app | raise exc | File "/root/faster-whisper-server/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 51, in wrapped_app | await app(scope, receive, sender) | File "/root/faster-whisper-server/.venv/lib/python3.12/site-packages/starlette/routing.py", line 715, in call | await self.middleware_stack(scope, receive, send) | File "/root/faster-whisper-server/.venv/lib/python3.12/site-packages/starlette/routing.py", line 735, in app | await route.handle(scope, receive, send) | File "/root/faster-whisper-server/.venv/lib/python3.12/site-packages/starlette/routing.py", line 362, in handle | await self.app(scope, receive, send) | File "/root/faster-whisper-server/.venv/lib/python3.12/site-packages/starlette/routing.py", line 95, in app | await wrap_app_handling_exceptions(app, session)(scope, receive, send) | File "/root/faster-whisper-server/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 62, in wrapped_app | raise exc | File "/root/faster-whisper-server/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 51, in wrapped_app | await app(scope, receive, sender) | File "/root/faster-whisper-server/.venv/lib/python3.12/site-packages/starlette/routing.py", line 93, in app | await func(session) | File "/root/faster-whisper-server/.venv/lib/python3.12/site-packages/fastapi/routing.py", line 383, in app | await dependant.call(**solved_result.values) | File "/root/faster-whisper-server/faster_whisper_server/routers/stt.py", line 286, in transcribe_stream | async with asyncio.TaskGroup() as tg: | ^^^^^^^^^^^^^^^^^^^ | File "/usr/lib/python3.12/asyncio/taskgroups.py", line 145, in aexit | raise me from None | ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception) +-+---------------- 1 ---------------- | Traceback (most recent call last): | File "/root/faster-whisper-server/.venv/lib/python3.12/site-packages/websockets/legacy/protocol.py", line 1301, in close_connection | await self.transfer_data_task | File "/root/faster-whisper-server/.venv/lib/python3.12/site-packages/websockets/legacy/protocol.py", line 974, in transfer_data | await asyncio.shield(self._put_message_waiter) | asyncio.exceptions.CancelledError | | The above exception was the direct cause of the following exception: | | Traceback (most recent call last): | File "/root/faster-whisper-server/.venv/lib/python3.12/site-packages/uvicorn/protocols/websockets/websockets_impl.py", line 332, in asgi_send | await self.send(data) # type: ignore[arg-type] | ^^^^^^^^^^^^^^^^^^^^^ | File "/root/faster-whisper-server/.venv/lib/python3.12/site-packages/websockets/legacy/protocol.py", line 635, in send | await self.ensure_open() | File "/root/faster-whisper-server/.venv/lib/python3.12/site-packages/websockets/legacy/protocol.py", line 948, in ensure_open | raise self.connection_closed_exc() | websockets.exceptions.ConnectionClosedError: sent 1011 (internal error) keepalive ping timeout; no close frame received | | The above exception was the direct cause of the following exception: | | Traceback (most recent call last): | File "/root/faster-whisper-server/.venv/lib/python3.12/site-packages/starlette/websockets.py", line 85, in send | await self._send(message) | File "/root/faster-whisper-server/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 48, in sender | await send(message) | File "/root/faster-whisper-server/.venv/lib/python3.12/site-packages/uvicorn/protocols/websockets/websockets_impl.py", line 345, in asgi_send | raise ClientDisconnected from exc | uvicorn.protocols.utils.ClientDisconnected | | During handling of the above exception, another exception occurred: | | Traceback (most recent call last): | File "/root/faster-whisper-server/faster_whisper_server/routers/stt.py", line 296, in transcribe_stream | await ws.send_json(CreateTranscriptionResponseJson.from_transcription(transcription).model_dump()) | File "/root/faster-whisper-server/.venv/lib/python3.12/site-packages/starlette/websockets.py", line 175, in send_json | await self.send({"type": "websocket.send", "text": text}) | File "/root/faster-whisper-server/.venv/lib/python3.12/site-packages/starlette/websockets.py", line 88, in send | raise WebSocketDisconnect(code=1006) | starlette.websockets.WebSocketDisconnect +------------------------------------ INFO: connection closed INFO: 10.224.0.4:13045 - "GET / HTTP/1.0" 200 OK

bjornbracko avatar Jan 07 '25 13:01 bjornbracko

I am having the same issue. The websocket gets disconnected every now and then. My logs look very similar:

ERROR:    Exception in ASGI application
  + Exception Group Traceback (most recent call last):
  |   File "/root/faster-whisper-server/.venv/lib/python3.12/site-packages/uvicorn/protocols/websockets/websockets_impl.py", line 244, in run_asgi
  |     result = await self.app(self.scope, self.asgi_receive, self.asgi_send)  # type: ignore[func-returns-value]
  |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  |   File "/root/faster-whisper-server/.venv/lib/python3.12/site-packages/uvicorn/middleware/proxy_headers.py", line 70, in _call_
  |     return await self.app(scope, receive, send)
  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  |   File "/root/faster-whisper-server/.venv/lib/python3.12/site-packages/fastapi/applications.py", line 1054, in _call_
  |     await super()._call_(scope, receive, send)
  |   File "/root/faster-whisper-server/.venv/lib/python3.12/site-packages/starlette/applications.py", line 113, in _call_
  |     await self.middleware_stack(scope, receive, send)
  |   File "/root/faster-whisper-server/.venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 152, in _call_
  |     await self.app(scope, receive, send)
  |   File "/root/faster-whisper-server/.venv/lib/python3.12/site-packages/starlette/middleware/exceptions.py", line 62, in _call_
  |     await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  |   File "/root/faster-whisper-server/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 62, in wrapped_app
  |     raise exc
  |   File "/root/faster-whisper-server/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 51, in wrapped_app
  |     await app(scope, receive, sender)
  |   File "/root/faster-whisper-server/.venv/lib/python3.12/site-packages/starlette/routing.py", line 715, in _call_
  |     await self.middleware_stack(scope, receive, send)
  |   File "/root/faster-whisper-server/.venv/lib/python3.12/site-packages/starlette/routing.py", line 735, in app
  |     await route.handle(scope, receive, send)
  |   File "/root/faster-whisper-server/.venv/lib/python3.12/site-packages/starlette/routing.py", line 362, in handle
  |     await self.app(scope, receive, send)
  |   File "/root/faster-whisper-server/.venv/lib/python3.12/site-packages/starlette/routing.py", line 95, in app
  |     await wrap_app_handling_exceptions(app, session)(scope, receive, send)
  |   File "/root/faster-whisper-server/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 62, in wrapped_app
  |     raise exc
  |   File "/root/faster-whisper-server/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 51, in wrapped_app
  |     await app(scope, receive, sender)
  |   File "/root/faster-whisper-server/.venv/lib/python3.12/site-packages/starlette/routing.py", line 93, in app
  |     await func(session)
  |   File "/root/faster-whisper-server/.venv/lib/python3.12/site-packages/fastapi/routing.py", line 383, in app
  |     await dependant.call(**solved_result.values)
  |   File "/root/faster-whisper-server/faster_whisper_server/routers/stt.py", line 286, in transcribe_stream
  |     async with asyncio.TaskGroup() as tg:
  |                ^^^^^^^^^^^^^^^^^^^
  |   File "/usr/lib/python3.12/asyncio/taskgroups.py", line 145, in _aexit_
  |     raise me from None
  | ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
  +-+---------------- 1 ----------------
    | Traceback (most recent call last):
    |   File "/root/faster-whisper-server/.venv/lib/python3.12/site-packages/websockets/legacy/protocol.py", line 1301, in close_connection
    |     await self.transfer_data_task
    |   File "/root/faster-whisper-server/.venv/lib/python3.12/site-packages/websockets/legacy/protocol.py", line 974, in transfer_data
    |     await asyncio.shield(self._put_message_waiter)
    | asyncio.exceptions.CancelledError
    | 
    | The above exception was the direct cause of the following exception:
    | 
    | Traceback (most recent call last):
    |   File "/root/faster-whisper-server/.venv/lib/python3.12/site-packages/uvicorn/protocols/websockets/websockets_impl.py", line 332, in asgi_send
    |     await self.send(data)  # type: ignore[arg-type]
    |     ^^^^^^^^^^^^^^^^^^^^^
    |   File "/root/faster-whisper-server/.venv/lib/python3.12/site-packages/websockets/legacy/protocol.py", line 635, in send
    |     await self.ensure_open()
    |   File "/root/faster-whisper-server/.venv/lib/python3.12/site-packages/websockets/legacy/protocol.py", line 948, in ensure_open
    |     raise self.connection_closed_exc()
    | websockets.exceptions.ConnectionClosedError: sent 1011 (internal error) keepalive ping timeout; no close frame received
    | 
    | The above exception was the direct cause of the following exception:
    | 
    | Traceback (most recent call last):
    |   File "/root/faster-whisper-server/.venv/lib/python3.12/site-packages/starlette/websockets.py", line 85, in send
    |     await self._send(message)
    |   File "/root/faster-whisper-server/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 48, in sender
    |     await send(message)
    |   File "/root/faster-whisper-server/.venv/lib/python3.12/site-packages/uvicorn/protocols/websockets/websockets_impl.py", line 345, in asgi_send
    |     raise ClientDisconnected from exc
    | uvicorn.protocols.utils.ClientDisconnected
    | 
    | During handling of the above exception, another exception occurred:
    | 
    | Traceback (most recent call last):
    |   File "/root/faster-whisper-server/faster_whisper_server/routers/stt.py", line 296, in transcribe_stream
    |     await ws.send_json(CreateTranscriptionResponseJson.from_transcription(transcription).model_dump())
    |   File "/root/faster-whisper-server/.venv/lib/python3.12/site-packages/starlette/websockets.py", line 175, in send_json
    |     await self.send({"type": "websocket.send", "text": text})
    |   File "/root/faster-whisper-server/.venv/lib/python3.12/site-packages/starlette/websockets.py", line 88, in send
    |     raise WebSocketDisconnect(code=1006)
    | starlette.websockets.WebSocketDisconnect
    +------------------------------------
INFO:     connection closed

Oudwins avatar Jan 09 '25 09:01 Oudwins

It might be related to this config: https://github.com/fedirz/faster-whisper-server/blob/master/src/faster_whisper_server/config.py#L244

Not sure why it's required - a long living websocket would be great to have, but then according to the settings, the VAD window must be infinite as well?

utrack avatar Jan 11 '25 09:01 utrack

I don't think that config is the issue. Because due to this disconnection I lose a part of the transcription. It cuts off and starts back up (I have reconnection built in) and I have logs so I know I am not dropping any packets during the reconnection process

Oudwins avatar Jan 12 '25 14:01 Oudwins

I am facing the same problem. Not sure if this helps, but a reproducible snippet below. The code captures microphone input and sends it to the transcription server. It manages to send captured audio, transcribes and the script receives the json transcript for a while. Then the connection breaks for some reason I have not been able to figure out...

import pyaudio
import asyncio
import websockets
import json

# Audio parameters
CHUNK = 1024  # Number of audio frames per buffer
FORMAT = pyaudio.paInt16  # Audio format (16-bit PCM)
CHANNELS = 1  # Mono audio
RATE = 16000  # Sample rate (16 kHz)

async def send_audio_and_receive():
    uri = "ws://192.168.1.118:8000/v1/audio/transcriptions"  # WebSocket server URI

    # Initialize PyAudio for audio recording
    p = pyaudio.PyAudio()
    stream = p.open(format=FORMAT,
                    channels=CHANNELS,
                    rate=RATE,
                    input=True,
                    frames_per_buffer=CHUNK)

    print("Connecting to the server...")

    async with websockets.connect(uri, ping_interval=20, ping_timeout=None) as websocket:
        print("Connected to the server. Start speaking...")

        try:
            while True:
                print("Read audio data from the microphone")
                audio_data = stream.read(CHUNK)

                print("Send audio data to the server")
                await websocket.send(audio_data)

                print("Receive and print any messages from the server")
                try:
                    response = await asyncio.wait_for(websocket.recv(), timeout=0.1)
                    print(f"Received from server: {json.loads(response)}")
                except asyncio.TimeoutError:
                    print("No message received within the timeout period")
                    continue

        except KeyboardInterrupt:
            print("\nStopped recording.")
        except Exception as e:
            print(f"An error occurred: {e}")
        finally:
            print("entered finally block")
            stream.stop_stream()
            stream.close()
            p.terminate()

if __name__ == "__main__":
    asyncio.run(send_audio_and_receive())

The log from the speaches container when the connection closes:

01 17 321: INFO: speaches.transcriber: audio _ transcriber: 68: Audio transcriber finished
01 17 20:16:56, 321: DEBUG: speaches.modeI_manager:_decrement_ref:81:Decremented ref count for self
01 17 20:16:56, 321: INFO:speaches.modeI_manager:_decrement_ref:84:ModeI is idle, scheduling offload in 300s
01 17 321: INFO: speaches . routers . stt : transcribe _ stream: 351: Closing the connection.
connection closed

It seems like the connection remains open until no speech is detected for at least 5 seconds. It then flushes and drops the connection. This behaviour is described in the inactivity_window_seconds setting parameter. However, what is the proposed way of keeping a persistent connection. Should the connection be reestablished here or the inactivity window be set very high?

oyhel avatar Jan 17 '25 20:01 oyhel