Using mixer to play 2 MP3 files at the same time?

[CircuitSetup]

I'm trying to get 2 MP3s to play at the same time using the mixer, but they either come out very distorted or things crash when playing a second sound. Is there something I'm missing or is this just not possible with MP3s?

The code looks like this:

#include <SPIFFS.h>
#include <AudioOutputI2S.h>
#include <AudioFileSourceSPIFFS.h>
#include <AudioGeneratorMP3.h>
#include <AudioOutputMixer.h>

AudioGeneratorMP3 *mp3 = new AudioGeneratorMP3();
AudioGeneratorMP3 *beep = new AudioGeneratorMP3();
AudioFileSourceSPIFFS *file[2];
AudioOutputI2S *out = new AudioOutputI2S(0, 0, 32, 0);
AudioOutputMixer *mixer;
AudioOutputMixerStub *stub[2];

void setup() {
    audioLogger = &Serial;
    out->SetOutputModeMono(true);
    out->SetPinout(I2S_BCLK, I2S_LRCLK, I2S_DIN);
    mixer = new AudioOutputMixer(32, out);

    play_file("/startup.mp3", 0.1, 0, true);
}

void loop() {

    time_loop()
    if (mp3->isRunning()) {
        if (!mp3->loop()) {
            mp3->stop();
            stub[0]->stop();
            stub[0]->flush();
            out->flush();
        }
    }
    if (beep->isRunning()) {
        if (!beep->loop()) {
            beep->stop();
            stub[1]->stop();
            stub[1]->flush();
            out->flush();
        }
    }
}

void play_file(const char *audio_file, float volume, int channel, bool firstStart) {
    Serial.printf("CH:");
    Serial.print(channel);
    Serial.printf("  Playing <");
    Serial.print(audio_file);
    Serial.printf("> at volume :");
    Serial.println(volume);

    if (!firstStart) {
        //cant do this if playing the first file after startup
        delete stub[channel];
        delete file[channel];
        if (channel == 0) delete mp3;
        if (channel == 1) delete beep;
        firstStart = false;
    }
    stub[channel] = mixer->NewInput();
    stub[channel]->SetGain(volume);

    file[channel] = new AudioFileSourceSPIFFS(audio_file);

    if (channel == 0) {
        mp3 = new AudioGeneratorMP3();
        mp3->begin(file[0], stub[0]);
    } else {
        beep = new AudioGeneratorMP3();
        beep->begin(file[1], stub[1]);
    }
}

void play_keypad_sound(char key) {
    if (key) {
        beepOn = false;
        if (key == '0') play_file("/Dtmf-0.mp3", 0.1, 0, false);
        if (key == '1') play_file("/Dtmf-1.mp3", 0.1, 0, false);
        if (key == '2') play_file("/Dtmf-2.mp3", 0.1, 0, false);
        if (key == '3') play_file("/Dtmf-3.mp3", 0.1, 0, false);
        if (key == '4') play_file("/Dtmf-4.mp3", 0.1, 0, false);
        if (key == '5') play_file("/Dtmf-5.mp3", 0.1, 0, false);
        if (key == '6') play_file("/Dtmf-6.mp3", 0.1, 0, false);
        if (key == '7') play_file("/Dtmf-7.mp3", 0.1, 0, false);
        if (key == '8') play_file("/Dtmf-8.mp3", 0.1, 0, false);
        if (key == '9') play_file("/Dtmf-9.mp3", 0.1, 0, false);
    }
}

void time_loop() {
    DateTime dt = rtc.now();
    y = digitalRead(SECONDS_IN);
    if (y != x) {      // different on half second
        if (y == 0) {  // flash colon on half seconds, lit on start of second
            // Do this on previous minute:59
            if (dt.minute() == 59) {
                minPrev = 0;
            } else {
                minPrev = dt.minute() + 1;
            }
            // plays every second
            play_file("/beep.mp3", 0.05, 1, false); 
        }
     }
}

[earlephilhower]

If you're just making DTMF, doing MP3s is about 10,000x overkill. Look at the AudioGeneratorFunction() for generating the 2 tones for each digit at 0.0001% the CPU and RAM usage.

Your crash may be related to deleting things that are stll part of the stubs. The mixer is a limited function one and not something like you would have on a real PC where it would, on an I2S-DMA-done IRQ basis, combine separate output buffers to a final output one.

[CircuitSetup]

Thanks for replying @earlephilhower

I'll definitely take a look at that library for generating the DTMF sounds.

I played around with it some more, removing the delete stub, and got it working without crashing, but it would not play 2 sounds at once either. I also tried combining the AudioGeneratorMP3 into one instance, but it crashes right away (which is why it is separated into 2 to begin with). Any idea why that would be happening?

[obdevel]

The mp3->loop() method blocks whilst there is data available from the source, which practically means the entire file. I have experimented with limiting it to a max number of samples per invocation (e.g. 100) so that control is returned to the program. This means you can get on with other things whilst the file is playing out, so long as you call mp3->loop() frequently.

Like you, I'm still trying to work out how to mix multiple concurrent streams.

---

Edit: the MixerSample can be made to work as expected by a simple edit to the generator class source, e.g. AudioGeneratorWAV.cpp.

At the top of the ::loop method add:

  const int max_samples = 100;
  int i = 0;

Then in the while conditional, break out of the loop once max_samples have been processed:

  ++i;
} while (running && i < max_samples && output->ConsumeSample(lastSample));

No changes are required to the example sketch itself. The first sample starts to play, then 3 secs later the second sample starts and the two streams are mixed, as expected.

More experimentation required, but I wonder if the generator loop methods could have an override or a default argument to set the max samples per loop.

[CircuitSetup]

@obdevel thanks for the insight. I didn't realize the loop was blocking. I'll have to play around with that some more.

@earlephilhower I looked at the AudioGenerator functions and I'm not seeing the one that actually generates tones. Can you point me in the right direction? In this case it needs to generate 2 tones at once for the DTMF.

[earlephilhower]

Look at PlayWaveFromFunction.ino. You would just implement the appropriate sine(t) for both frequencies and sum them up.