Record output of audio visualizer with proper delays in audio player

[BenBergman]

I'm trying to record the output of an audio visualizer I built that uses HTML5 audio tag sources for input and feeds in through an analyzer node. When played without CCapture it runs just fine at a high framerate. With CCapture running it runs at a slow frame rate (as expected) but the audio is still running real time. This causes the video output to appear as though it is in fast forward.

I see that CCapture overrides HTMLAudioElement.prototype.currentTime. Do I need to enable anything to get this working? I have tried with Firefox and Chrome and I don't see any errors in the console with verbose mode on.

[spite]

Yes, unfortunately there's no way to "throttle" the internal workings of Web Audio, so the AnalyserNode will run in real time, desyncing from the canvas capture. That's a problem that affects all captures that deal with FFT and signal processing, I've run into that many times. There's only two real solutions or workarounds: reimplement the AnalyserNode so it runs throttled, or pre-generate and store the FFT result. 😐

[BenBergman]

Ah, interesting. I'm not even using the FFT, I'm just looking at the waveform. If Web Audio can't be throttled, what effect does swapping out the currentTime property have? Is there an alternative to using AnalyserNode for just looking at the waveform which works with CCapture?

[spite]

CurrentTime is part of the HTMLAudio/VideoElement, so it's monkey patched in case your code is syncing to an audio or video file.

There isn't a way to sync AnalyserNode, AFAIK, except the two options i mentioned before, replacing frequency domain data (FFT) with time domain data in your case.

I mean, technically stopping and starting the audio graph should work, but i've never tried...

[BenBergman]

Ah, gotcha. currentTime is just for time based sync with tracks, not actual reactivity to sound. I thought maybe it was used internally for playback coordination or something but was failing to do so when using analyserNode. My mistake.

For the purposes of my current project's proof of concept I've managed to get the frame dropping with a screen capture program to a tolerable level. If I want to improve this further I'll make two apps: one that analyses the audio and spits out data with time stamps and another that feeds that data into the visualizer and renders out using CCapture, much as you suggested.

Thanks for the feedback!