Example of simultaneous playback/record of large files on ASIO device

[bradricketts]

Hello,

I'm not sure if this is the best place to ask about this, but I'm having a great deal of difficulty getting simultaneous playback and record to work with Python buffer objects rather than the numpy array functions. Those functions are really nice, but the files I'm going to be working with are very large, and memory usage will be a problem. Is there an example somewhere of how to do this with sounddevice?

I found an example here; https://pythonexample.com/snippet/pastreampy_tgarc_python

Based on that, this is my latest try https://www.dropbox.com/s/9kqt7ccnkvjdtx1/PlaybackCapture.py?dl=0 I guess there is a size limitation on Github as I couldn't get the code to paste correctly in this text box.

It does playback, but nothing is recorded. I'm somewhat of a novice at this, so I'm having some trouble figuring out what's wrong, or if I should just scrap this and write something myself.

Thank you

[tgarc]

Hi @bradricketts, that code example got indexed from some arbitrary time in my account..I never posted that code and that particular version was a WIP that may not be completely working. I would suggest starting from a minimum working example. Have you seen rec_unlimited.py in the examples? I would probably start from there and modify it to use native python buffer objects instead of numpy arrays. Once you get that working it should be simple enough to modify it to include playback as well. From personal experience I would suggest using memoryviews and/or bytearrays as an alternative to numpy arrays.

If you have more specific questions about implementation feel free to ask.

[mgeier]

@bradricketts Sure, it's no problem to ask questions here!

I'm having a great deal of difficulty getting simultaneous playback and record to work with Python buffer objects rather than the numpy array functions. Those functions are really nice, but the files I'm going to be working with are very large, and memory usage will be a problem.

If you have very large amounts of data, you should split it into pieces. You can do this with both the NumPy functions and the Python buffer functions. You make it sound like the problem is NumPy, but it isn't!

The problem is that sounddevice.playrec() insists on using a single buffer in memory for all recorded data. You should use sounddevice.Stream with a custom callback function where you will get the input/output data of the current block as NumPy arrays. You can still, if you don't want to use NumPy for some reason (which is also OK), use a sounddevice.RawStream.

As @tgarc has suggested, you should have a look at the rec_unlimited.py example.

And there is even a second example that does the other part: play_long_file.py! Incidentally, this one uses Python buffers.

When you combine the two, you can choose either NumPy or Python buffers, it's up to you. I think with NumPy it's easier. And you can always change it to Python buffers once it is working with NumPy arrays!

There are a few things that are notably different between the recording and the playback examples:

For playback ...

• ... it will be easier to implement if you disallow blocksize=0.

• ... you'll need to somehow limit the size of the queue or the speed in which you write data from the file to the queue. I've used the maxsize argument for that. If you don't limit it, you might end up writing the whole file at once into the queue ...

• ... you'll have two different sizes to consider: the size of one audio block and the size of the queue, i.e. the maximum number of audio blocks stored in the queue at one time. If either one is too short, different bad things are going to happen. In the callback function of my example I'm trying to handle this.

• ... you should probably pre-fill the queue in the beginning, in order to make sure the first call of the callback function has some data to play out. I'm doing that in my example.

If you combine playback and recording, you will have two sf.SoundFile objects. You can use them in the same with statement or in two nested with statements, I think it doesn't really matter.

Finally, I think you will be able to combine the two scripts without having to create additional threads. But if you want to include this into some bigger piece of code, it might make sense to create (an) additional thread(s) for reading from and/or writing to a file.

If you are satisfied with your combined examples, you could make a pull request and we can add it to the examples!

[bradricketts]

Thank you both for the excellent responses, I greatly appreciate it. Unfortunately I've been pulled onto another project so I won't be working on this for another few days, but when I do come up with some better examples I will definitely add them with a pull request.

[mgeier]

@bradricketts Any news?

[benoitvalery]

Also interested in recording microphone while playing arbitrary wav files. I'm actually trying to merge the two mentioned example to suit my needs (though I'm not familiar with Stream-like objects).

[mgeier]

@benoitvalery Do you have a concrete problem I might be able to help you with?

[benoitvalery]

@mgeier Yes actually. In an experimental context, I'm trying to use the sounddevice library to simultaneously record the microphone and play a small wave file repeatedly (e.g., 'beep beep beep beep...'). The problem is, though I have a good level in Python, I'm not familiar at all with Streams. I understood I could create two separate stream, one for each need (input & output), but 'till now, I could not reach a satisfying result. I would need a minimal example of 'recording while playing' to adapt it to my needs. I could also paste a bit of my code, but I have to simplify it. Thanks for your reaction.

[mgeier]

@benoitvalery You should not use multiple streams unless you have a good reason to.

It sounds like a single stream should be enough for your situation. Then you'll also have a single callback function. This callback function has to make sure that your "beep beep beep" is written to the output buffer at the right time. And it also has to be prepared to record the input buffer whenever it is supposed to.

If the duration of your recordings is known in advance, you can simply create a NumPy array and record into it. If the duration is not known, I would suggest using a queue.

It might be easier to implement playback and recording separately as a first step, and once both work, simply combine the two callback functions into one.

[benoitvalery]

@mgeier Thanks for your advice. I succeed in implementing continuous microphone recording using an InputStream and a queue. However, output seems a bit more tricky. Suppose I have a very short wav file ('beep.wav'). I want to play it each second (sep = 1). Here is what I tried. The only thing I get is a kind of continuous buzz (like a continuous electronic noise). Any idea ? Thanks for your help !

import time
import soundfile as sf
import sounddevice as sd

sep = 1
sf_beep = sf.SoundFile('beep.wav')

def callback_read(outdata, frames, time, status):
    if (time.currentTime - begin_time) % sep == 0.0:
            outdata[:] = beep_data

begin_time = time.time()
while True:
    with sd.OutputStream(samplerate=44100, device=18,
                                        channels=2, callback=callback_read):
        beep_data = sf_beep.buffer_read(1024, dtype='int16')

[fatchild]

This might be something people are after. Simultaneous playback and recording of a generated wave.

My requirements:

• Play a signal out into my test amplifier
• Record the signal simultaneously
• Do some basic correlation

import sounddevice as sd
import numpy as np

duration = 10  # seconds
fs=44100
dtype='int16'
channels=channels
device_out = 9
device_in = 5
amplitude = 0.2
frequency = 1000

start_idx = 0

samplerate = sd.query_devices(device_out, 'output')['default_samplerate']

def callback(indata, outdata, frames, time, status):
    # Deal with in the recording
    volume_norm = np.linalg.norm(indata)*10
    print("|" * int(volume_norm))

    # Deal with the output
    global start_idx
    t = (start_idx + np.arange(frames)) / samplerate
    t = t.reshape(-1, 1)
    outdata[:] = amplitude * np.sin(2 * np.pi * frequency * t)
    start_idx += frames




with sd.Stream(device=(device_in, device_out), callback=callback, channels=1, samplerate=samplerate):
    sd.sleep(duration * 1000) 

Sounddevice is an awesome python module. So many possibilities, thanks!

[mgeier]

@fatchild Thanks for the example! Your variable dtype is unused, but if you would actually use dtype='int16' as argument to sd.Stream(), your example wouldn't work as expected!

@benoitvalery You are creating, starting and closing streams very rapidly in a loop. That's not good. You should create a single stream. When you start the stream, the callback function will automatically called repeatedly. Also, you should not compare Python's time.time() with the time.currentTime you get in the callback function. Those are different timebases. If you want to get the time outside the callback function, you should use https://python-sounddevice.readthedocs.io/en/0.3.14/api.html#sounddevice.Stream.time. But it might actually be simpler to just count the number of frames that have already been played. This way, you don't have to convert between seconds and frames. Depending on your exact situation, it might also be helpful to use a queue to send information (about audio data to be played back) to the callback function.

[fatchild]

Thanks for your comments, i'm working on a new improved example. As part of my project I want to be create a function with sounddevice which...

• opens a stream
• writes data to the output (sinewave)
• generates the indata as the peak value and yields it in realtime back to another function for analysis

I know the first two points are not hard to do as I've done with my previous example. But where would the yield of the processed indata go? would it be in the callback.

q = queue.Queue()
def PlayRec(self, duration=10, channels=1, amplitude=0.5, frequency=1000, test_signal='sinewave'):

        def whitenoise(indata, outdata, frames, time, status):
            # Deal with in the recording
            volume_norm = np.linalg.norm(indata)*10
            q.put(indata.copy())
            # Deal with the output
            mean, std, num_samples = 0, 1, frames
            samples = np.random.normal(mean, std, size=num_samples)
            samples = samples.reshape(-1, 1)
            outdata[:] = amplitude * samples


        with sd.Stream(device=(AudioTools.input_device_index, AudioTools.output_device_index), callback=test_signal, channels=channels, samplerate=samplerate) as stream:
            # Generate output data stream for processing within chosen method
            #sd.sleep(duration * 1000)
            input()

Second function

    def test_yield(self):
        for indata in PlayRec(AudioTools, test_signal='whitenoise'):
            while True:
                yield q.get()

There's a code snippit of what i'm working on along with the second function, seems to do something, but returning a form of ndarray different to that of rec()

[fatchild]

I have just seen the read() write() functions instead of using a callback... this could be a possibility too.

[mgeier]

@fatchild You can of course try to use the .read() and .write() methods, but I think they are very limited. I normally prefer using callback functions.

With regards to getting information out of the callback function, I would suggest using a queue (as you already showed in your example).

You should have a close look at the examples:

• plot_input.py uses a queue to send a (heavily downsampled) copy of the input signal to be plotted by Matplotlib.
• rec_gui.py even has two queues: one for the audio data (to be recorded in a separate thread) and one for peak values (to be displayed in the GUI). Latter could be similar to what you want to do in your example, right?

I don't quite understand what you want to do with the generator function in your example, and how iterating over PlayRec(...) is supposed to work. Can you please clarify?

[mgeier]

@fatchild Any news?

Does anyone else in this thread still have unanswered questions?

[benoitvalery]

@mgeier No thanks. No more issue for me.

[fatchild]

@mgeier No thankyou. I worked a solution.

Unfortunately I am no longer working on the project. But from what I remember, most of my issues came down to setting the blocksize correctly as I was working on a Pi. So for anyone who has an issue for simultaneous record and playback on a Pi; it is possible, just check the blocksize in the stream and don't be afraid to make it quite high.

[ryanckim]

Hello Matthias,

I happened to work on exactly the same issue as @fatchild did - playing back a wave file and capturing the mic input as another wav file (for correlation-based analysis). Probably I'll move on to using a signal generated on the fly instead of the file playback, but I thought for now I could simply combine the two examples: rec_unlimited.py and play_long_file.py. Starting from play_long_file.py, what I've done is nothing more than

1. using indata inside the callback definition:

def callback(indata, outdata, frames, time, status):
    # assert frames == blocksize
    if status.output_underflow:
        print('Output underflow: increase blocksize?', file=sys.stderr)
        raise sd.CallbackAbort
    assert not status
    try:
        data = qOut.get_nowait()
    except queue.Empty:
        print('Buffer is empty: increase buffersize?', file=sys.stderr)
        raise sd.CallbackAbort
    if len(data) < len(outdata):
        outdata[:len(data)] = data
        outdata[len(data):] = b'\x00' * (len(outdata) - len(data))
        raise sd.CallbackStop
    else:
        outdata[:] = data
    offsetInOut = time.inputBufferAdcTime-time.outputBufferDacTime
    # Place incoming data to fileRec queue - is this correct?
    qIn.put(indata[:])

2. placing the input file opening/writing functions with the stream:

try:
    with sf.SoundFile('../1kHz_square_wave.wav') as fileStim, sf.SoundFile('../testRecordingPython.wav', mode='x', samplerate=fileStim.samplerate, channels=1, subtype='FLOAT') as fileRec:
        for _ in range(buffersize):
            data = fileStim.buffer_read(blocksize, dtype='float32')
            if not data:
                break
            qOut.put_nowait(data)  # Pre-fill queue
        stream = sd.RawStream(blocksize=blocksize, channels=1, dtype='float32', callback=callback, finished_callback=event.set)
        with stream:
            timeout = blocksize * buffersize / fileStim.samplerate
            print('timeout:', timeout)
            while data:
                data = fileStim.buffer_read(blocksize, dtype='float32')
                qOut.put(data, timeout=timeout)
                fileRec.buffer_write(qIn.get())        # grab input buffer and write to fileRec - is this the right place?
            event.wait()  # Wait until playback is finished

I'm not a regular Python user so I wasn't sure where I should have put this qIn.put(indata[:]) inside the callback, and the fileRec.buffer_write(qIn.get()) with my stream. Could you please help? I think the use of indata[:] might not be right either - any hint would be really appreciated.

And I'm also trying to get the offset by that line inside my callback offsetInOut = time.inputBufferAdcTime-time.outputBufferDacTime which I would like to use to align my output/input files for correlation. Is this the right approach?

Thanks!

[mgeier]

I wasn't sure where I should have put this qIn.put(indata[:]) inside the callback,

This doesn't really matter. Only in the case where the callback is stopped, you might move it before the raise statement.

and the fileRec.buffer_write(qIn.get()) with my stream.

The place you have put it looks OK, except that when data is empty in the end, there will still be a few blocks in qIn, which you should obtain in yet another loop.

It's important to pre-fill the queue for this to work, but you are doing that anyway in the example code.

I think the use of indata[:] might not be right either - any hint would be really appreciated.

Indeed, it's not quite right. Your use of indata[:] creates a separate NumPy array, but it is re-using the same memory (which may subsequently be overwritten by future callback invocations). You should use indata.copy() instead (as in the rec_unlimited.py example).

And I'm also trying to get the offset by that line inside my callback offsetInOut = time.inputBufferAdcTime-time.outputBufferDacTime which I would like to use to align my output/input files for correlation. Is this the right approach?

This is a possible approach, but depending on the host API, the calculated time might not be very accurate. Some devices even return 0.0 all the time, which would lead to an offset of 0.0.

Even if the times are sufficiently accurate, they still don't contain the latency of the DAC/ADC on your audio hardware.

Ideally, you should measure the latency with a physical loopback cable.