High latency in USBVideo stream via SSH due to playbin3 buffering

[hamztouiz]

Hello,

I’m currently using Labgrid’s USBVideoDriver to stream a remote USB camera.

The stream works, but the latency is extremely high (around 30 seconds).
The exporter and coordinator run on the same machine as the USB camera, and the client PC (running the viewer) is on the same LAN.
To rule out network issues, I tested the link with an scp transfer of a large file and achieved 112.6 MB/s, so the network bandwidth is not the bottleneck.

After investigating, I found that the latency completely disappears when I replace the default playbin3-based RX pipeline with a manual GStreamer pipeline:

rx_cmd = [
    "gst-launch-1.0",
    "fdsrc",
    "!",
    "jpegdec",
    "!",
    "queue", "max-size-buffers=1", "leaky=downstream",
    "!",
    "videoconvert",
    "!",
    "autovideosink",
    "sync=false",
]

With this RX pipeline, the latency becomes near zero.
This makes me think that playbin3 might be introducing buffering or synchronization that is not suitable for low-latency SSH transport.

Questions

1. Is this behavior expected when using playbin3 in this context?
2. Is there an existing way in Labgrid to override the RX pipeline?
3. As a workaround I created my own driver, is this the recommended approach, or should this be handled inside USBVideoDriver?

Thanks in advance for the guidance.

[jluebbe]

@tretter @michaelolbrich @KarlK90 Do you have any good ideas? Stefan did the switch from playbin to playbin3 in 65998f685cbdadac10339a7c49034c548a7a6589 / https://github.com/labgrid-project/labgrid/pull/1336.

[michaelolbrich]

I'm don't understand how the pipeline above can work at all. Form what I see in the code, the video is muxed with a matroskamux element at the sender side. So just a jpegdec cannot work. Any other changes?

[tretter]

@hamztouiz Which labgrid version are you using?

https://github.com/labgrid-project/labgrid/commit/a9295fca1f5e605073b7754604ec1cc1d01766c1 removes some internal buffering in the playbin3 pipeline to reduce the latency of the receiver pipeline.

[hamztouiz]

@tretter I am using 25.0.1 version.

@michaelolbrich I just modified the rx pipeline as described above. I did not modify the TX pipeline. For the camera model I have, the caps are “image/jpeg,width=1280,height=720,framerate=30/1,” if that helps.