Ringbuffer underflow/overflow

[fotisdr]

I decided to make an issue here to summarize my issues/requests with rtmixer, perhaps for future fixes/updates. In general, what I am trying to do is to execute an audio processing algorithm in real-time using rtmixer. I've already implemented this with jack-client, but, since rtmixer is better for many reasons, I'm trying to do the same with rtmixer.

• As discussed previously in the jack-client repo, my main issue is that, in cases where my processing algorithm is too slow, the audio stops. The cause is that the ringbuffer is empty (underflow) and the playback action is removed by the rtmixer: https://github.com/spatialaudio/python-rtmixer/blob/d7095f228f58d08fa69916d3657e4996526144cd/src/rtmixer.c#L388-L393 The thing here is that this happens even in cases where it shouldn't. What I mean is that, for a blocksize of 1024 samples and a sampling rate of 16 kHz for instance, the processing algorithm should have 60 ms available at the worst case. However, with a processing algorithm that takes about 40-50 ms I always get a ringbuffer underflow (I also put a printf inside the rtmixer.c if to make sure this was the cause). In fact, I measured the required time for the processing inside python and the worst case is that it takes 51 ms to execute. However, the audio playback always stops (ringbuffer underflows) and the 'weird' thing is that it always happens after 4 frames for my script (maybe it's not weird, I just haven't understood the reason yet). You can find the framework of my script at the end of this issue. On the other hand, this is solved if I double the pre-filling of the queue, but that increases latency and in my case is the last solution. I also tried increasing the latency from 'low' to 'high' and also the MixerAndRecorder blocksize to 0 but nothing helped (in fact setting blocksize=0 gives immediately a ringbuffer underflow and I still haven't found the cause). Still, what's troubling me is whether the buffer underflow makes sense for a processing algorithm that is at least 10 ms faster than the limitation. From various timings that I did there seems to be some delay each time in the filling of the input queue with the first frame, which may be causing this buffer underflow (I haven't found the exact reason yet). However, if I increase the pre-filling (also for the input queue) and set the blocksize=0 it works. Of course if a slow processing for a frame occurs it stops (because of a buffer underflow), but that makes sense as we said. Correct me if got this wrong but, by setting blocksize=0, no matter how much I pre-fill the queues I would assume that after a while the playback catches up to the case where no pre-fill existed (the latency difference is lost).
• Another thing that would be really good is if there was a feature added to the API that could allow for the playback to continue after ringbuffer overflows/underflows. My C programming skills are not that great so I wasn't able to check somehow the buffer underflows and implement this from scratch, but I guess it would make sense to have a flag in the python API that will allow ringbuffer over/underflow in the rtmixer playback.

#!/usr/bin/env python

from __future__ import division, print_function
from time import time,sleep
import rtmixer
import sys
import numpy as np

blocksize = 1024

latency = 'low'
samplerate = 16000
channels = 1
qin_size = 4
q_size = 4*qin_size

stream = rtmixer.MixerAndRecorder(
    channels=channels, blocksize=blocksize, #blocksize=0?
    latency=latency, samplerate=samplerate)
with stream:
    print('  input latency:', stream.latency[0])
    print(' output latency:', stream.latency[1])
    print('            sum:', sum(stream.latency))
    print('requested delay:',timeout)

    samplesize = 4
    assert {samplesize} == set(stream.samplesize)

    qin = rtmixer.RingBuffer(samplesize * channels, qin_size * blocksize)
    record_action = stream.record_ringbuffer(qin)

    q = rtmixer.RingBuffer(samplesize * channels, q_size * blocksize)
    buffer = np.zeros((blocksize,1),dtype='float32') # or q_size*blocksize?
    q.write(buffer)
    play_action = stream.play_ringbuffer(q)

    try:
        while True:
            while qin.read_available < blocksize:
                if record_action not in stream.actions:
                    break
                sleep(0.001)
            if record_action not in stream.actions:
                break
            read, buf1, buf2 = qin.get_read_buffers(blocksize)
            t = time()
            buffer = np.frombuffer(buf1, dtype='float32')
            noisy[0,:,0] = buffer
            # processing of 'noisy' is performed here and 'clean' is computed
            buffer = clean.ravel() #.astype('float32')
            qin.advance_read_index(blocksize)
            while q.write_available < blocksize:
                if play_action not in stream.actions:
                    print('Ringbuffer underflow')
                    break
                sleep(0.001)
            if play_action not in stream.actions:
                print('Ringbuffer underflow')
                break
            q.write(buffer)
            print(time()-t) # measure processing time
    except KeyboardInterrupt:
        print('\nInterrupted by User')