howler.js icon indicating copy to clipboard operation
howler.js copied to clipboard

Rendered overlays

Open apc518 opened this issue 2 years ago • 0 comments

Feature Proposal

Enable combining two or more sounds into one rendered sound.

Purpose: performance

I will make a PR if this is something that sounds like it could be accepted. Actually I might make a fork in any case lol.

While multiple sounds can be played at once, when you start trying to play hundreds or thousands of sounds, things quickly break. A solution to this that those of us in music production know all too well is "bouncing", aka "rendering in place" or "freezing" tracks. That is, taking many sounds and rendering them together so you dont have the computational load of playing N tracks, but just one.

It doesn't seem like the audio buffers are directly accessible via any methods or properties of Howl objects so this doesnt seem possible at the moment with Howls, but even if it is possible, it'd be better if this was an intentional feature.

I'm envisioning something either like pydub's AudioSegment.overlay method or a Howl.plus method that accepts another Howl as an argument perhaps along with offset or volume multiplier arguments, and which returns a brand new Howl, leaving the first two unchanged. Then of course some helper method to do this with N sounds.

Possible Implementation

this is all pseudo-ish code (syntactically correct but with simplifications)

an overlay method like:

overlay: function(self, otherHowl, offset, volume) {
    for (let i = 0; i + offset < self.buffer.length && i < otherHowl.buffer.length; i++) {
        self.buffer[i + offset] += otherHowl.buffer[i] * volume;
    }
}

and/or a plus method (could have _overlay if you only want plus in the API)

plus: function(self, otherHowl, offset, volume) {
   let copy = self.copy();
   self.overlay(otherHowl, offset, volume);
   return copy;
}

then a method to do this for arbitrarily many sounds

renderTogether(howls){
   // add them all together
   let longestHowl = howls.reduce((h1, h2) => h1._duration > h2._duration ? h1 : h2).copy();
   for (let howl of howls){
      longestHowl.overlay(howl);
   }
   
   // normalize the result, could make this optional
   let maxSample = longestHowl.buffer.reduce((a, b) => Math.abs(a) > Math.abs(b) ? a : b);
   for (let i = 0; i < longestHowl.buffer.length; i++){
      longestHowl.buffer[i] /= Math.abs(maxSample);
   }

   return longestHowl;
}

apc518 avatar Aug 08 '22 00:08 apc518