Using Peaks.js without waveform data

[obliwetator]

I work with very large audio recordings (3hr+) and because of that it's not always possible to have the waveform data generated in a timely manner. It's also impossible when the recording has not finished. Also the data for such large files takes a while to download. My question is whether it is possible to use dummy data just to be able to scrub the timeline while the actual data is being generated/downloaded?

[chrisn]

Initialising without waveform data currently isn't possible, but I guess it could be made to work by providing the duration, sample rate, and zoom level (samples per pixel), then use setSource when the real data is downloaded.

[chrisn]

Another issue is that resampling the waveform data (for the overview waveform) blocks user interaction. If you look at a performance trace, how much time is spent resampling and how much waiting for the waveform data file to download?

[obliwetator]

The library validates the that the data is valid (length and sample must be correct) so I wasn't able to to make it work, at least with the JSON version (I haven't mannualy created the binary version since I believe those checks are also there.). For now I am using a very basic player and just load the library when everything is done loading. For a 5~ hour long recording the .dat file is about ~6MB. Once it is downloaded the waveform loads in about ~500ms. I measured it with performance.now().

[obliwetator]

Also, I guess it's possible to manually create the data client side if you know the duration of the audio file but I haven't tried it yet.

[chrisn]

I've just done a couple of tests. Here's a performance trace with a 28MB waveform, which is taking about 400ms to download the .dat file (on localhost) then 1.2 seconds to show the waveform:

The resampling can be avoided if you don't use an overview waveform, and if the initial zoom level in the zoomview is the same as the waveform data.

Next, I tried creating the data client side. I added code to create an empty waveform with the same length:

const length = 14969670;
const headerSize = 20;
const buffer = new ArrayBuffer(length * 2 + headerSize)
const view = new DataView(buffer);

view.setInt32(0, 1, true); // Version
view.setUint32(4, 1, true); // Flags
view.setInt32(8, 44100, true); // Sample rate
view.setInt32(12, 8, true); // Samples per pixel
view.setInt32(16, 14969670, true); // Length

and then use these options in Peaks.init() instead of a dataUri object:

waveformData: {
  arraybuffer: buffer
}

and then call setSource() after Peaks.init() to fetch the real .dat file.

But this takes a similar amount of time again with the empty waveform, because of the resampling:

and because the resampling is done on the UI thread, the user's not going to be able to interact during that time.

One solution could be to pre-compute the overview waveform server-side and add a way to pass this in to Peaks.js, so you could quickly show the overview waveform, then fetch the .dat file for the zoomable waveform.

Or we could try moving the resample to a worker, to avoid blocking the UI thread. But this either involves copying the waveform to the worker (may be slow?) or using SharedArrayBuffer.

[chrisn]

Another approach is to change Peaks.init() to take the audio sample rate and waveform samples per pixel values. These should be enough to initially draw the time axis labels and then the waveform added when it's ready.

[mikkelwf]

Any update on this? Being able to render the waveform without actually loading the audio source would be much appreciated

[chrisn]

@mikkelwf I haven't tried this, but it may be possible to do this already by passing in a custom player object (see doc/customizing.md) when you call Peaks.init(), where the player object doesn't do anything.