An audio call with one person not hearing the other person for a quite a while

[schildbach]

Describe the bug An audio call with one person not hearing the other person for a quite a while

Server

• Version: [1.8.4]
• Environment: [Docker]
• LiveKit is used via Matrix/Element Call

Client Latest versions of Element X for Android

Expected behavior Clear audio at all times.

Additional context We used audio only (cam disabled).

I'll attach the LiveKit logfile to this issue.

[schildbach]

livekit-bad-call.log

[schildbach]

In the log, there are a couple of "WARN large sequence number gap" that coincide with the loss of bidirectional audio.

And then, "ERROR livekit service/signal.go:183 could not handle new participant".

[boks1971]

Yeah, it seems like client has stopped sending audio for a while. If you look at that "large sequence number gap" log, you will see a field called "timeSinceHighest". That is showing 7s+ in the first log of its kind. Can you please check that the client does not have a network brown out? Looks like client is not sending audio for a while.

[schildbach]

A network brownout could be the reason, but it's a bit hard to tell after the fact. We were both maintaining other network connections (online games) at the same time without noticable problem.

I have a question about this event, judging by the timestamps only differing by a millisecond, could be the same event:

2025-04-05T20:36:39.217Z       rtpstats/rtpstats_receiver.go   "participant": "@…:KUJTXOMATP"   "timeSinceHighest": "7.398291784s"
2025-04-05T20:36:39.218Z       rtpstats/rtpstats_sender.go     "participant": "@…:EVKOOWOHVB"   "timeSinceHighest": "7.398291784s"

What does this mean exactly? The first line seems to imply that packages from client/session KUJTXOMATP to LiveKit were lost, the second line seems to imply packages could not be sent from LiveKit to EVKOOWOHVB. I'd like to find out, which of the two, or if both connections were troubled. In other words: is it the home Internet connection of one of us, or is it the server?

I'm running LiveKit in Docker, here is its config (no config file, just env vars):

  livekit:
    image: livekit:1.8.4
    read_only: true
    cap_drop:
      - ALL
    security_opt:
      - "no-new-privileges"
    secrets:
      - livekit_keys
    environment:
      LIVEKIT_KEY_FILE: /run/secrets/livekit_keys
      LIVEKIT_PORT: 7880
      LIVEKIT_RTC_UDP_PORT_START: 7882
      LIVEKIT_RTC_UDP_PORT_END: 7892
      LIVEKIT_RTC_USE_EXTERNAL_IP: "false"
    ports:
      - 7882-7892:7882-7892/udp

I don't yet use host networking, because I don't want to expose the control port 7880 to the internet – it should only be accessible via reverse proxy which does SSL and stuff. Could proxying UDP ports through "dockerd" cause such problem? It's Docker version 20.10.24+dfsg1 that comes with Debian 12.

I have increased UDP buffer size to the value LiveKit asked for on first startup:

# sysctl -a | grep net.core.rmem
net.core.rmem_default = 5000000
net.core.rmem_max = 5000000

[boks1971]

The logs 1ms apart is because the SFU is just like a router. It is taking media packets coming in from a participant and sending it to all the other participants in the room. So, the rtpstats_receiver.go log is saying that the audio publishing participant is not sending data for a while. The rtpstats_sender.go log is a knock on effect because of the gap in the sender's data. That pair of logs shows an issue in upstream from client -> server. If there were multiple participants in the room, each one would log in rtpstats_sender.go as all of them are missing the same data from the publisher.

is it the home Internet connection of one of us, or is it the server? This is hard to say, but it is in the path from publishing client -> server. Typically, it is on client side unless the server is not set up properly.

I am not sure if proxying UDP ports through dockerd would be problematic. Are there any network interface drops you are seeing there?

The socket buffer memory size looks fine.

[schildbach]

Thanks for your thorough reply.

Are there any network interface drops you are seeing there?

Well have an eye on that, but I think Wifi was fine. Otherwise the person wouldn't have been able to navigate in the game she was in at the time of the issue.