waveform-data.js icon indicating copy to clipboard operation
waveform-data.js copied to clipboard

Create Waveform Data Web Audio Node

Open davidturissini opened this issue 6 years ago • 0 comments

Similar to https://github.com/bbc/waveform-data.js/issues/46, but I think a better approach is to write implement this as a Web Audio Node. Here is what I propose:

const waveformDataNode = createWaveformDataNode(audioContext, { scale: 512 }, (fullWaveform, sampleWaveform) => {
    // This callback gets two arguments:
    // Argument 1 (fullWaveform): Is the full waveform for every sample that has been processed
    // Argument 2 (sampleWaveform): Is the waveform only for the current sample.
});

anyWebAudioNode.connect(waveformDataNode);
waveformNode.connect(anyOtherWebAudioNode);

// API call to start gathering Waveform data as it happens, passthrough otherwise
waveformNode.beginRender();

Couple thoughts:

  1. By implementing this as a web audio node (ScriptProcessorNode or possible AudioWorklet), we can generate a waveform from any audio data that can be processed via Web Audio API. That means that we can get waveform data from attached gain nodes and microphone streams.

  2. This approach works with either a regular "AudioContext" or an "OfflineAudioContext".

  3. My initial implementation was just a "container" Waveform that held an array of other Waveforms. This may or may not be the best way, but I don't have a lot of experience (or use cases) for zooming in and out. Some help on this front would be much appreciated @chrisn

  4. There are some tasks that need to be done before this can be production ready, most notably this: https://github.com/bbc/waveform-data.js/issues/61. We simply cannot spin up a new Worker instance for every sample. It's very inefficient and has caused my browser to spin many times.

davidturissini avatar Jan 12 '19 05:01 davidturissini