bigbluebutton
Audio and Video not in sync in BBB 2.6 [Live Meeting]
Describe the bug In the German BBB Adopter's meeting we discussed a degradation from 2.5 to 2.6 with audio and video not being in sync anymore during meetings. We could also reproduce this bug at random in this meeting. It, however, seemed to only show up between specific users (User A says something. This is perfectly in sync for User B but not for User C). At least five admins regularly experience video and audio not always being in sync during meetings on their respective installations. All these installations have set up fullAudio (for 2.6) and didn't experience this asynchronism in their 2.5 setup.
To Reproduce Unfortunately we don't really know how to reproduce this bug since it shows up sporadically. We are also not aware if fullAudio might be the cause of this. This can also be reproduced when resetting the jitterbuffer as suggested here: https://github.com/bigbluebutton/docker/issues/237
Expected behavior Video and Audio do not drift.
Actual behavior Video and audio do drift.
Additional context Although we are aware that this is an unspecific bug report we still decided to file it since it's not specific to one person's setup. If you have any idea on what we might change in our setup we are willing to try.
To be specific, is this an issue with live meetings, or with recordings of meetings?
If this is only recordings, you might be seeing #18268
Because of the way BigBlueButton does separate streams for audio and video, there's unfortunately no guarantee that the audio+video for any given participant will be in sync in the live meeting. It depends a lot on the internet of specific people, and can be caused either on the side of the person sharing media, or the person viewing it.
Hi, it's about live meetings. We filed this bug because we didn't experience this issue in BBB 2.5. May it be related to the fullAudio setup?
We could also reproduce this bug at random in this meeting. It, however, seemed to only show up between specific users (User A says something. This is perfectly in sync for User B but not for User C).
We suspect the problem is audio. When you have a user that is not in sync, can you have them drop video and share it again (we suspect the sync issue will remain), and then have them drop audio and share it again. Does the video now sync?
may be hardware dependency?
we do not have such reports from servers with older cpu. like Xeon CPU E31240 but have reports for server: Xeon E-2286G
@ffdixon the problem about your proposal is that (at least on our server) video and audio is not out of sync all the time. It usually takes a couple of seconds (idk, I think about 15s) until it's in sync again. This usually repeats a couple of times during meetings. Starting/Stopping audio/video as you proposed is not that easy to do since we almost can't rule out that it came back in sync all by itself.
@amg-web I don't think it's a hardware dependency since it ran on the exact same VMs with BBB 2.5 installed before without problems.
Good news.
It looks like the FreeSWITCH team recently merged (two weeks ago) improvements to the sync in audio when using OPUS and a Jitter buffer. See: https://github.com/signalwire/freeswitch/pull/2069
The pull request states:
We have been using the jitter buffer of freeswitch (audio only) with some webrtc users, it works fine, but I noticed it was not accelerating/shrinking, unless facing a jitter buffer reset.
Once we noticed that, we decided to cap to maximum size to 120ms. (Some users were complaining about excessive delay.) While looking at the WebRTC jitter buffer stats, I noticed it was sometimes reaching a size of 300ms and more.
We're going to update 2.7-beta to use this latest build of FreeSWITCH and, if it improves the sync, we'll back port it to 2.6.
If this is only recordings, you might be seeing #18268
Sorry to jump on this with a slightly unrelated bug (mine is a recording issue). But am curious as to which issue you're referring to @kepstin? (The one you linked to is this same issue).
I can reproduce this error with the latest version of BBB (2.6.10). We are using fullAudio.
The corresponding recording issue should be https://github.com/bigbluebutton/bigbluebutton/issues/18131
Good news.
It looks like the FreeSWITCH team recently merged (two weeks ago) improvements to the sync in audio when using OPUS and a Jitter buffer. See: signalwire/freeswitch#2069
The pull request states:
We have been using the jitter buffer of freeswitch (audio only) with some webrtc users, it works fine, but I noticed it was not accelerating/shrinking, unless facing a jitter buffer reset. Once we noticed that, we decided to cap to maximum size to 120ms. (Some users were complaining about excessive delay.) While looking at the WebRTC jitter buffer stats, I noticed it was sometimes reaching a size of 300ms and more.
We're going to update 2.7-beta to use this latest build of FreeSWITCH and, if it improves the sync, we'll back port it to 2.6.
Hey, any chance this could be back ported to 2.5 too? We're having similar issues and we're not ready to move to 2.6 yet :-/
Hi, do you have any news on this issue? It doesn't seem to be merged to 2.7 (https://github.com/bigbluebutton/bigbluebutton/pull/18300)? I'm glad to help if testing or similar is needed :-)
removed as I thought you were talking about a different (somewhat related) issue
Unfortunately we don't really know how to reproduce this bug since it shows up sporadically. We are also not aware if fullAudio might be the cause of this. This can also be reproduced when resetting the jitterbuffer as suggested here:
@rob0403 if you have a reproducible scenario at hand and are able to, please verify whether the desync happens with 2.6's default bridge as well (sip). It would speed things up when I get to this.
Hi, do you have any news on this issue? It doesn't seem to be merged to 2.7 (#18300)? I'm glad to help if testing or similar is needed :-)
@rob0403 Also: as far as the proposed upstream FreeSWITCH changes to jitterbuffer acceleration are concerned, they may improve the situation - pending further testing. If you're willing to try it, I can provide instructions on how to build a patched FreeSWITCH package with the aforementioned changes (or I can provide the package itself) alongside setup instructions. That would also speed things up.
Even if the aforementioned jitterbuffer changes improve the situation, I still want to figure out a couple of things:
- why it "regressed" in your scenario between 2.5 and 2.6
- if it is specific to the
fullaudiobridge or all bridges (this one is very important)
This also applies to the other interested/affected parties here (cc @mwuttke et al)
Hi, I just tried to reproduce the error with a colleague that had a huge desync between video/audio on friday in the same setup as we had on friday. Today, however, we were unable to reproduce it (both on 2.6.10, fullAudio and 2.6.11, fullAudio).
My feeling is that the desync especially occurs on Firefox. I'll keep on trying to reproduce the error and keep this issue updated. Thanks for your feedback. I'll come back at building the FreeSWITCH package as soon as I better know how to reproduce the desync.
Hi, I just tried to reproduce the error with a colleague that had a huge desync between video/audio on friday in the same setup as we had on friday. Today, however, we were unable to reproduce it (both on 2.6.10, fullAudio and 2.6.11, fullAudio).
My feeling is that the desync especially occurs on Firefox. I'll keep on trying to reproduce the error and keep this issue updated. Thanks for your feedback. I'll come back at building the FreeSWITCH package as soon as I better know how to reproduce the desync.
@rob0403 Thanks for trying to reproduce.
I'm setting up a bunch of lab scenarios to try and reproduce this - I'll update this issue when I've found something.
As for the tests I asked, please fwd the following info of the problematic user when you are able to reproduce:
- Browser (with version)
- OS (with version)
- Connection type (carrier (3G/4G/5G) | cabled + wi-fi | cabled + ethernet | satellite + something, ...)
- Microphone input device (rough description - I'm interested on whether it's an USB, Bluetooth [or other wireless protocol], P2/10 or integrated microphone)
- Whether there was a change in input and/or output device, at the system level, prior to or during the call
Just a short update on this until I can reproduce:
- Browser: I only noticed it by now with Firefox users. But I don't remember what browser @danimo was using in the BBB adopter's meeting?
- OS: Happened on macOS and Windows. And probably on some linux flavour (@danimo?)
- Connection type: cabled + wifi
- Microphone input device: I noticed this on USB microphones and on notebook internal microphones
- Change in input and/or output device -> not aware of
I'll keep you updated. Thanks for diving into this.
Update: We can't reproduce the huge drift we sometimes experience in the following scenarios:
- All Users use Chrome/Chromium + fullAudio
- All Users use Firefox + sipjs
- All Users use Chrome/Chromium + sipjs
We can reproduce the huge drift in the following scenario:
- All Users use Firefox + fullAudio
Additional information about the reproducible scenario: We had five users. Four of them used Windows (Firefox 116.0.02), one used Linux (Firefox 115. 1.0esr). The drift occurs from time to time with a huge interval between audio/video. A couple of seconds later all is fine again. I hope this helps.
@rob0403 Great info, this narrows things down. Thanks for the reply. Can you confirm two more things:
- whether
forceRelayOnFirefoxis true in bbb-html5.yml - are you using external coturn instances or the one bbb-install provides alongside the BBB server?
Hi @prlanzarin ,
forceRelayOnFirefox : false
coturn setup:
- haproxy and coturn on vm. stun is not activated but only: turns:hostname:443?transport=tcp
with hostname as the URL of the machine
This is the case for both setups (fullAudio + sipjs)
FYI the aforementioned FreeSWITCH changes landed in 2.6.12/2.7-rc.1, so testing is welcome. I'm still not happy with it - the allegations that this might be a regression between 2.5 and 2.6 go beyond that patch (even if it "fixes the issue"), so I'm still looking into it (in dribs and drabs, since time is short unfortunately).
I still can reproduce the drift with the latest version of BBB (2.6.12). We are using forceRelayOnFirefox: false. and external coturn instances. The user with the drift detected used the firefox browser and wifi.
We could reproduce it, too (on 2.6.12.) a) The drift, however, is not as big as it was before and it feels like it resynchronizes earlier than before (which means that the situation overall improved) b) I have the feeling that virtual backgrounds worsen the drift.
@mwuttke do you experience this, too?
I have same problem with 3.0.0-alpha.7 (1118).
If there is a problem with two streams, maybe solution will be to switch to video+audio in on stream when user enable video? And when it disable video, then fallback to only audio stream?
I'am very new to BBB so I'am not sure it is possible.
I have same problem with 3.0.0-alpha.7 (1118).
If there is a problem with two streams, maybe solution will be to switch to video+audio in on stream when user enable video? And when it disable video, then fallback to only audio stream?
I'am very new to BBB so I'am not sure it is possible.
That's precisely the solution, but not a simple one due to how things are currently organized. The plan is to add experimental support to a new media framework in 3.0 that should improve this, so stay stuned.
we have issue with this started Since 24th September. it was ok before this date. bbb 2.7.13 which logs to collect?