baresip icon indicating copy to clipboard operation
baresip copied to clipboard

jbuf frame completeness

Open cspiel1 opened this issue 2 years ago • 8 comments

This PR improves jbuf in order to

  • ensure frame completeness for video and correct order of frames,
  • reduce the buffered frames with return EAGAIN if it is possible, but not too early.
  • Correct order of frames/packets for audio streams.
  • Does not increase O-notation complexity for jbuf_put() nor jbuf_get().
  • Makes adaptive mode obsolete and removes it.

This PR still is a draft and will be compared to https://github.com/baresip/baresip/pull/2757.

cspiel1 avatar Oct 06 '23 10:10 cspiel1

PR description reads:

config: remove jbuf mode adaptive

but docs/examples/config still has it for audio and video.

Have I misunderstood something?

juha-h avatar Oct 06 '23 10:10 juha-h

Plots for video

  • without packet loss jbuf-video
  • with packet loss jbuf-video-loss

cspiel1 avatar Oct 06 '23 11:10 cspiel1

Plots for audio

The second plot of each set is the ajb plot of the adaptive audio buffer. See https://github.com/baresip/re/blob/main/rem/aubuf/ajb.c !

Underruns in jbuf lead to underruns in the aubuf. Sometimes this can't be avoided, e.g. if there is a packet lost.

  • min size 1 without packet loss jbuf-audio ajb
  • min size 1 with packet loss jbuf-audio-loss ajb-loss
  • min size 0 without packet loss jbuf-audio-zero ajb-zero
  • min size 0 with packet loss jbuf-audio-zero-loss ajb-zero-loss

cspiel1 avatar Oct 06 '23 11:10 cspiel1

Looks like for audio, the buffers does not play well together, if I see correctly we have 2-3 packets in jbuf (20-60ms) + up to 80ms within aubuf and this does not compensate a 50ms jitter without packet loss (leads to real aubuf underruns)?

sreimers avatar Oct 06 '23 12:10 sreimers

Looking at the ajb plots auf aubuf it looks like that the computed jitter does not increase fast enough to the expected value of 50ms. Ideally we would expect that there only underruns at the beginning of the simulated jitter. For these plots I used this jitter simulation:

sudo tc qdisc add dev ifb1 root netem delay 0ms 50ms loss 0.7%

Note that ajb is only active if configured with audio_buffer_mode adaptive.

This PR should touch only jbuf and

  • ensure frame completeness for video and correct order of frames.
  • Reduce the buffered frames with return EAGAIN if it is possible, but not too early.

The jbuf plots show that underruns because of out-of-order packets occur only at the start of the jitter. Additionally if there is a real packet loss. This is all we could expect from jbuf how it is used currently.

After this we could think about moving ajb from aubuf to jbuf and use a timer for jbuf_get() and the decoding.

cspiel1 avatar Oct 09 '23 05:10 cspiel1

please resolve the conflicts ...

alfredh avatar Oct 26 '23 07:10 alfredh

I will resolve the conflicts tomorrow. @sreimers and me plan to compare this with: https://github.com/baresip/baresip/pull/2757

cspiel1 avatar Oct 26 '23 17:10 cspiel1

please update the description with more details.

Can you please also rebase this on top of git HEAD ?

alfredh avatar Dec 02 '23 12:12 alfredh