Seeking very slow

[merlinaudio]

Hi! Amazing lib, thanks for building this :)

I wanted to see if there's any way I can seek in realtime.

I'm building a video editor, so the user should be able to seek/scrub between specific frames. However, the seek functions are slow. I have an MP4 that isn't all intra-frames. So i assume what makes it slow is that this library needs to seek up to an intra-frame, then apply deltas to get to the frame i want.

Here's what I tried/want to try:

• Use the seek functions (too slow)
• Cache all VideoFrames (too much memory usage)
• Cache VideoFrames near users playhead (couldn't make this work properly; requires creating lots of readers)
• Use Proxy files (DNxHD/ProRes) through WASM (haven't tried, probably super hard to implement)
• Create an intermediate MP4 where every frame is an intra-frame (haven't tried, not sure how)

Using a reader (readVideoPacket + genEncodedVideoChunk) is fast enough for realtime playback, but only allows seeking forward.

Is there anything obvious I'm missing here? How do other video editors or realtime players that use WebCodecs do this?

[ForeverSc]

It seems a bit strange that seeking only supports backward i-frames. Could you provide an MP4 file where seeking is too slow? I will also add support for seeking any frame, allowing you to retrieve the GOP and use WebCodecs to decode p-frames or b-frames. You can refer to the library WebAV, which uses mp4box.js and WebCodecs to extract the GOP and decode.

[puny-d]

Thanks your project :), any plan about seeking any frame ?

[ForeverSc]

Thanks your project :), any plan about seeking any frame ?

@puny-d I just released version 2.2.0, which now supports seeking any frame! :)

[merlinaudio]

Wow, I'm gonna check it out asap @ForeverSc thank you so much for building and maintaining this!!!