More than 2 concurrent websocket-connections cause problems for FFmpeg

[Trollgeir]

Whenever I try to open more than 2 browsers to stream the websocket connections they seem to not work. After not too long I get a broken pipe warning from FFmpeg. I added an extra print in the AudioProcessor() at the BrokenPipeError for debugging with print(self.ffmpeg_process.stderr.read()):

ffmpeg version 6.1.1-3ubuntu5 Copyright (c) 2000-2023 the FFmpeg developers\n built with gcc 13 (Ubuntu 13.2.0-23ubuntu3)\n configuration: --prefix=/usr --extra-version=3ubuntu5 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --disable-omx --enable-gnutls --enable-libaom --enable-libass --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libglslang --enable-libgme --enable-libgsm --enable-libharfbuzz --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-openal --enable-opencl --enable-opengl --disable-sndio --enable-libvpl --disable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-ladspa --enable-libbluray --enable-libjack --enable-libpulse --enable-librabbitmq --enable-librist --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libx264 --enable-libzmq --enable-libzvbi --enable-lv2 --enable-sdl2 --enable-libplacebo --enable-librav1e --enable-pocketsphinx --enable-librsvg --enable-libjxl --enable-shared\n libavutil 58. 29.100 / 58. 29.100\n libavcodec 60. 31.102 / 60. 31.102\n libavformat 60. 16.100 / 60. 16.100\n libavdevice 60. 3.100 / 60. 3.100\n libavfilter 9. 12.100 / 9. 12.100\n libswscale 7. 5.100 / 7. 5.100\n libswresample 4. 12.100 / 4. 12.100\n libpostproc 57. 3.100 / 57. 3.100\n[matroska,webm @ 0x604acab80840] EBML header parsing failed\n[in#0 @ 0x604acab80740] Error opening input: Invalid data found when processing input\nError opening input file pipe:0.\nError opening input files: Invalid data found when processing input\n'

[QuentinFuxa]

Thank you for the issue.

Do you get the error for the 2 streams or just for one ? A ffmpeg pipeline (AudioProcessor()) is created for each client, so I have to understand how interferences can occur

[Trollgeir]

The first two connections I make seem fairly stable. Whatever browser window I add to record after that seems to be flaky, and they struggle to keep up with the transcription backend. The error messages is from the backend relates to the connections I make after the first two, and they eventually stop and loop-crash forever while the first two seems to keep going.

I've tried with both large and tiny models as well just to rule out model-inference issues, in addition to adding more workers to uvicorn and disabling reload.

Also confirmed that the number of separate FFmpeg processes running on my system (attached to the AudioProcessor()) were the same number of connections.