Do video cutting in Studio directly, not just on the Opencast side

[LukasKalbertodt]

#618 introduced a simple video cutter that works by sending SMIL files to Opencast. This means that the cutting happens on the Opencast side and the "Download" button in Studio downloads the uncut video.

Cutting in the users browser is technically challenging though. I see a few possibilities here:

• Create hidden <video> element and play the video part that is kept in there, use captureStream() to get a stream that can be recorded in MediaRecorder again. We can either set the cut marker times manually to only show the part of the video that is kept or use media fragment URLs.
  • Pro: probably the easiest solution
  • Cons: the whole video is reencoded which takes lots of time and decreases the quality of the video. Futhermore, even on fast PCs, I am not sure if we can record with more than 1x video speed (i.e. if we speed up the <video> element, will the MediaRecorder also record more frames? Or will it just look like a sped up video?)
• Create a hidden canvas, manually render the video inside of it and record that.
  • Pro: we potentially have more control than over the simple <video> element
  • Cons: the whole video still needs to be fully reencoded and this is likely more tricky than the <video> solution
• Understand and decode the resulting webm file (required for #517 anyway) and try to only reencode the part from the cut point to the next key frame (or vica versa for the end cut). The reencoding is done via <video> or the <canvas> solution.
  • Pro: Only a small part of the video is reencoded, so it should be pretty fast and the video quality only decreases in the start and the end.
  • Cons: Probably the most complex solution. Furthermore, we now depend on the video container and video codec used by the browser, as we need to understand it (it's unlikely browser will drop support for webm and vp8, but we at least need to add code for the case that we don't understand the browser output).
• Compile ffmpeg (or something like that) to WASM and do the complete reencoding on our own.
  • Pro: independent of browser container/codec. We can also probably just reencode the start and end of the video to save time.
  • ffmpeg is probably fairly huge (in the MBs gzipped) and by far not all if its features are required. So an alternative or a subset would be required to keep the app size reasonable.

There is also some somewhat related discussion in https://github.com/elan-ev/opencast-studio/issues/454

[mtneug]

How hard is it to merge multiple clips? Can a browser do it without reencoding? Studio could record x sec. segments. Cutting would then mean to only merge a selection of them.

[LukasKalbertodt]

How hard is it to merge multiple clips? Can a browser do it without reencoding?

Mmhh depends. Assuming we can parse and construct custom WEBMs, it might be possible without reencoding.

But in your solution, the user could not cut very precisely, but just every x seconds, right? I don' think that's a practical solution :/

[mtneug]

But in your solution, the user could not cut very precisely, but just every x seconds, right? I don' think that's a practical solution :/

True, but I don't think that e.g. setting x=5s is unpractical. And even for longer x, similar to your third solution, it is possible to only reencode certain parts.

[LukasKalbertodt]

Ah ok, now I get it. So recording in 5s chunks is a solution to the "we need to find the next key frame" problem. So you still want to alllow frame-perfect cutting. That could be an interesting approach.

[afogel]

Not sure if you'd seen, but there's a new wasm port of ffmpeg for transcoding. I wonder how difficult it might be to modify for video manipulation within the browser: https://github.com/Etwas-Builders/modfy.video Anyway, thought I might post it just in case! I wish I had bandwidth to take it on and try to make PR 😭

[LukasKalbertodt]

@afogel That's very interesting, I haven't heard of that project yet. So thanks for bringing that to our attention. Do you happen to know how large the resulting wasm file is? That was my major concern regarding the "compile all of ffmpeg to wasm" idea.

Unfortunately, we also don't really have time to work on studio, particularly not this issue. But we'll see what we can do in the future!

[afogel]

(not sure why I didn't get notified, sorry for the late response!) It's pretty heavy still. I think they're planning on paring it down, so I'll monitor the project and let y'all know if something comes of it.