Use ffmpeg.js for offline recording

[clabe45]

A major issue with our current realtime recording approach is lag that occurs while recording can be seen in the render. In other cases, recording can be slower when it's done in realtime than when it's done offline.

As suggested by @lnaztm, we can use requestFrame() to combine all the image frames into a video blob. An offline audio context can be used with a separate media recorder to generate the audio blob. Then, we can use ffmpeg.js to merge this with the video into the final render.

Of course every image frame could be saved as a PNG in memory and then combined with the audio from the offline context with ffmepg.js, but I'm guessing that would use more memory.

If anyone can think of any improvements that can be made to this overall plan, please comment your ideas

[galipmedia]

https://github.com/spite/ccapture.js/ this is a library that records canvas at any frame rate without drops. For purely video without audio this may be helpful?

[clabe45]

@galipmedia sorry for the late reply. That library doesn't seem to support video formats other than webm, but it may fix the video stutters. Maybe we could add an experimental rendering mode that uses it

[clabe45]

We might be able to make recordings smoother with the ideas proposed in #114. Otherwise, we can try ffmpeg