diffusionstudio
Question about video memory usage / improvement
Hello,
I'm trying to support large video files (4k 60fps - multi-gig), but my concern is that will make the browser tab run out of memory because this library loads the entire video into memory when rendering. However, I see that when previewing the video on the page with attachPlayer, it appears to start playing almost immediately, with a document.createElement('video') in js controlling it.
My question: why not use this js video element to loop through its frames when rendering the video, instead of the current behavior which downloads the entire video into memory and extracts frames with the ffmpeg worker? My hope is that this would reduce memory usage because the video would be handled by the browser itself which can offload the video data onto disk / apply memory strategies. This lib currently loads the video into a blob which takes loads of memory.
Hopefully that makes sense. Please let me know your thoughts. Thank you!
This is certainly an issue at the moment. Unfortunately, the html video element is to slow and doesn’t support frame exact extraction. We're currently working on a solution based on WebCodecs that will lead to the same result. Should be available in 2-3 weeks.
