Large memory leak in p5.SoundFile

[bplmp]

When loading many p5.SoundFiles (more than 150 in my case) the browser becomes slow.

I've tried using the dispose() method or assigning the sound objects to null, but nothing seems to free up the memory after the sounds have been loaded. Any way to do this?

p5 forum topic for reference: https://forum.processing.org/two/discussion/comment/57306

[brig002]

same problem here. but in my case, just playing same sound file ~20 times in quick succession freezes browser (my sound file is less than a second). can happen even after sounds have stopped playing. reference post: https://forum.processing.org/two/discussion/14223/running-sound-play-more-than-20-times-slows-browser-to-a-crawl#latest

[therewasaguy]

Hey @bplmp and @brig002, thanks for reporting. I've been meaning to dig into this and I'm sorry I have not chimed in.

I'm pretty sure that the issue has to do with two things:

1. a potential memory leak in the SoundFile

2. The way that SoundFile keeps track of currentTime. This is not built in to the Web Audio API's BufferSourceNode, so it uses an additional ScriptProcessorNode to track the position in the buffer. That node is bad news and I think we have to get rid of it. I've noticed that the problems are particularly bad in firefox.

   Unfortunately, I could not find a better way to accurately track current time of a playback buffer in the web audio api in cases where the user changes the playback rate. For more, you can see this thread.

   I think the library should either drop support for SoundFile.currentTime, or make a note that this method will not work accurately if the playback rate is changed. If users want accurate playback, the documentation should encourage use of the p5.dom library's createAudio method to create an HTML5 Audio Element. That element is designed to track playback position, while Web Audio API is designed for sample-accurate scheduling. As part of this, it would be good to make a tutorial that outlines when to use which type of audio player...

[therewasaguy]

Looking into the memory issues, this is an informative thread. Excerpt:

"memory is large because the Web Audio API decodes your small MP3 into 32-bit LPCM – which will give you something on the order of 10MB per minute per channel"

@bplmp when you load 150 sound files, are any of them referencing the same file? Or are they all unique audio clips? How long are the clips?

[brig002]

hi @therewasaguy - i put my sample up here for you to check out: http://dfilm.com/lucs/vivarium/ click about 10 times within the vivarium to create 10 creatures... then as they bump into the wall, the same very short sound file plays repeatedly. it actually seems to work fine in safari and chrome, but brings FF to a halt very quickly.

[bplmp]

@therewasaguy , thanks a lot for looking into this.

The 150 sound files that I'm loading are all under 35kb or so each. I think none of them are longer than 2 seconds in duration. The same file can be loaded multiple times though (but never more than 10 times I think), that's something that could definitely be improved.

Check out a sample here that I'm running into slowing problems (click on the > to play): http://shipping-forecasting.herokuapp.com/latest.html?1 (some of the words in the sentence will not be played but that is expected behavior as I don't have yet sound files for them).

[therewasaguy]

@bplmp @brig002 can you try out the latest version of the library (here) and let me know if there are still problems?

[koincidence]

hi @therewasaguy , i followed with interest your discussion, being developing the same kind of project (with a bunch of around 100 different sound files playing together quite fast) and having the same slow down problem and also noisy playback.

I tested with different user's platforms : chrome is the best (quite prefect), safari is ok, firefox is very bad (slowing done immediately), iOS and android are okay on fast devices but play slowly and sound very noisy on low end devices, and in general the cpu performance and available memory seem to be critical. hope this helps…

Following your last advice, i tried to test with the latest version v0.3.0 of p5.sound.js I then noticed another (smaller) problem here : i am calling loadSound from preload with a completion callback, useful to trace which sound file is not present if any. This works with v0.2.16 of p5.sound.js but not with v0.2.17 and v0.3.0 : the callback is called correctly, but preload is endless and setup never comes… hereunder is a simple test sketch to show this problem sketch.txt

i would be happy to know if this issue can be fixed, but mainly i would like to know about any solution to the main problem of performance with loading and playing quickly a lot of sound files. thanks for all

[brig002]

Oh hey @therewasaguy - sorry, just saw the notification of your post above now. Just tried your new p5.sound.js but I'm getting an error with it. Funny that the error is in p5.js and not in the new sound library, but have confirmed that everything works with the old sound library and not with this new one.

TypeError: window[obj] is undefined return window[obj].prototype[func].apply(context, args); p5.js (line 5736, col 5)

[koincidence]

hi @therewasaguy , some new test result : i bypassed for now my preload / loadSound with callback problem and did a test with the v0.3.0 version of p5.sound.js : it works much better (although not perfect) with firefox, that doesn't slow down and stop playing as before. good news !

[therewasaguy]

@koincidence thanks for the feedback!

Using a callback in preload has caused issues in the past—it's best to just do one or the other (preload or callback). However, the issues were apparently fixed in https://github.com/processing/p5.js/pull/956. Which version of the p5 core library have you been using?

@brig002 can you post some example code, and which version of p5.js are you using?

[koincidence]

@therewasaguy thanks for your response i'm using p5.js v0.4.21 Jan 04,2016, quite recent. again, preload with callback worked with p5.sound.js v0.2.16...

[therewasaguy]

Ah, thanks @koincidence. It must have to do with the fact that loadSound now has both an error and success callback.

[koincidence]

okay, i added a second callback for error, and it works perfectly with p5.sound.js v0.3.0 the conclusion is that when loadSound used in preload, no or 2 callbacks are necessary, not one. thanks so much ! more feedback to come about overload performance tests...

[koincidence]

hi @therewasaguy , to go on with my remarks about performance, I opened a new issue : loading a great number of sound files gives a noisy rendering on low end devices #100

[brig002]

@therewasaguy, sorry for long delay. I'm not getting GitHub notifs. Just re-opened this thread b/c am working on a new project and having same issue. Using: /! p5.sound.js v0.3.0 2016-01-31 */ /! p5.js v0.4.23 March 04, 2016 */

Example here: http://dfilm.com/lucs/spooky_lamp/ Click to add more bugs. Works fine in Mac Chrome/Safari. In FF, it slows to a crawl with about 20 bugs.

-ben

[shadoof]

New to p5.js-sound, but have been working on a sonification and think I've found more about a related memory leak. I've made a simple test sketch here http://programmatology.shadoof.net/ritajs/p5sound-leak . This loads one SoundFile object (16 secs) and plays many times very quickly. Profiling shows that a 'Listener' is added at least every time the sound calls play(). See the attached graph from Chrome's dev tools with the number of Listeners growing steadily (and more or less corresponding to a playCount variable). If p5.js-sound is keeping track of current play() time (see earlier in this thread) then this memory leak may be related to an issue like this one https://stackoverflow.com/questions/48201104/listeners-in-chrome-dev-tools-performance-profiling-results . In my sketch, I don't want to dispose() of SoundFile objects but neither do I want them to make other objects that they don't dispose of when no longer needed (when they've finished playing).

[dhowe]

I am having the same issue. Has there been any further investigation into this? It makes p5.sound unusable except for the simplest sketches... @shadoof @therewasaguy

[swirly]

+1 here Same problem. Making falling balls, with a ping pong sound on bounce. After a while, firefox goes slow, CPU goes UP and memory explodes. You can reproduct the case with that script : https://editor.p5js.org/swirly/sketches/Bkv_QhVzN If you comment the sound parts, goes perfectly well If you use sounds, becomes sluggish after a while. There seems to be at least a memory leak. (same sound played a lot)

This is really a p5sound problem. I've recreated the same file use html5 audio, it works flawlessly https://editor.p5js.org/swirly/sketches/SkbI4iNfV P5js sound unusable for "real" project with this, only sample sound playback but no sound effect allowed with this

[dirkk0]

Unfortunately I have to agree. Although I like p5-sound very much, recognising this as a bug not in my code but in p5-sound cost me a lot of time in my current project.

On the bright side, I was able to switch to Tone.js for playback and effects, while the recording still is done with p5-sound.

I really wish this would be ironed out, though.

[dhowe]

I've also switched to Tone.js at this point. Works as expected

[brianpeiris]

Just wanted to note that this is still a problem in Firefox. I've tested with this simple example on both Ubuntu and Windows, with the currernt master branch, and I've found that if you call .play() ~500 times, the memory leak will just start running away on its own and consume all the computer's memory, even if you stop calling play.

function preload(){
  soundFile = loadSound('../files/doorbell.mp3');
}

function setup() {
  createCanvas(400,200);
}

var playInterval = null;
var counter = 0;
function mousePressed(){
  if (!playInterval) {
    soundFile.setVolume(0);
    playInterval = setInterval(() => {
      soundFile.play();
      counter++;
    }, 200);
  } else {
    clearInterval(playInterval);
    playInterval = null;
  }
}

function draw(){
  background(256);
  text('File is ready!  Click to pause / play.', 50, 10);
  text('Sound file played ' + counter + ' times. Check memory usage.', 50, 30);
}

[DeveloperChallenge]

Hi, @therewasaguy, I'm still facing the same issue. i.e memory leak, computer browser starts hang after some times when i played 6-7 songs and switch between them. But mobile browsers crashed after some time. I tried latest 1.0.0 version but seems still to have the same issue.

[Adrian-Serna]

Problem still persists with Version: 1.0.1 of 2021-06-11 :/