bigbluebutton
Rasos performance gain
This is really interesting -- can you be more specific on the improvements you encountered?
We run a few BBB instances, two of them under quite heavy load.
We've applied a low latency kernel as we always do with video / audio related projects, including some Jitsi videobridges.
For BBB, we never had an issue with freeswitch suddenly disappearing nor audio crackling sound.
We did not perform any load / selenium testing though to check how many hundred users / streams can be served. Bottleneck with NodeJS will remain, though.
To confirm, you you applied these changes to 18.04 using BigBlueButton 2.3-dev -- right? Was this on a bare-metal server or virtual server?
We have applied a low latency kernel both on BBB 2.2 on Ubuntu 16.04 and on BBB 2.3-dev on Ubuntu 18.04 (the latter not in production use yet). All our BBB servers run on bare metal.
@rasos did you experience the problems which this change is supposed to fix/improve on the same environment and same version of bbb before applying these changes? On paper, the low-latency kerne should make next to no difference for a typical BBB setup.
I'd suggest to do some reproducible testing and collect some data.
If we do end up making these changes, we should prefer to use the linux-lowlatency-hwe-16.04 kernel package in order to get the newer 4.15 kernel backported from ubuntu 18.04 instead of the 4.4 kernel from ubuntu 16.04.
A low latency kernel should have no (or even a negative) effect.
- The default scheduling frequency is 250Hz (4ms per tick), which is increased to 1000Hz (1ms per tick) for the low-latency kernel. Freeswitch processes audio in 20ms chunks and the jitterbuffer compensates delays of up to 120ms. Freeswitch also runs at higher priority, so if there is CPU available, freeswitch should have no issue processing audio in a timely manner. A higher scheduling frequency should have no positive effect here.
- Kernel preemption allows high priority interrupts to steal CPU from processes in the middle of a tick. This is important for real-time processing with extremely small deadlines. But Freeswitch does not need such realtime guarantees. Throughput is more important than low latency in most scenarios.
- The "audio crackling" issue is caused by packet loss, but this is usually caused by actual network issues and not by overflowing buffers within the network card or kernel. Increasing scheduling frequency or enabling preemption would not help at all. If your network buffers overflow because freeswitch does not read from them fast enough then you have serious throughput problems. Low latency kernels will make this worse, as they sacrifice throughput for low latency.
So, unless someone actually compares those two kernel types on identical hardware in a realistic load scenario with verifiable results, I would be very careful changing defaults. The low-latency kernel may do more harm than good.
This pull request is old and, I believe, most users are deploying BigBlueButton using the default kernels.