remotion icon indicating copy to clipboard operation
remotion copied to clipboard

Overlay mode

Open BenjaminLindberg opened this issue 2 years ago • 11 comments

@JonnyBurger 👋🏻 It would be really good if there was a "OverlayMode" where the composition gets rendered over a video. This would mean you don't need to have a video element in the composition lowering the performance. It could be an argument when you render with lambda or with ssr, you pass the videos URL/path or something like that. If this would be possible it would be a great feature for sure! 😄

BenjaminLindberg avatar Sep 05 '22 19:09 BenjaminLindberg

Good idea, I'm +1 on this!

JonnyBurger avatar Sep 06 '22 17:09 JonnyBurger

Not sure I'm understanding - so this to improve the performance of the player component?

UmungoBungo avatar Sep 06 '22 23:09 UmungoBungo

@UmungoBungo No, it is to make the render faster. A usecase that arises often (also in your app) is that you render something on top of a video, and instead of screenshotting a Video tag you would just render the overlay and then use FFMPEG to overlay it

JonnyBurger avatar Sep 08 '22 05:09 JonnyBurger

Any news regarding this?😁

BenjaminLindberg avatar Oct 28 '22 15:10 BenjaminLindberg

@B3nnjoe A bit reprioritized since one of the potential big customers has found a workaround and they don't necessarily need it anymore. If we would do it, we would implement it together with other features that require multiple FFMPEG passes, like #1276

JonnyBurger avatar Oct 31 '22 09:10 JonnyBurger

@JonnyBurger sounds good! But one thing im wondering is the workaround. Would it be possible to get a rough explanation?

BenjaminLindberg avatar Oct 31 '22 12:10 BenjaminLindberg

@B3nnjoe something similar can be achieved by rendering only a transparent overlay in ProRes format (https://www.remotion.dev/docs/transparent-videos/) and then use an FFMPEG command to combine both the original video and the overlay.

JonnyBurger avatar Oct 31 '22 14:10 JonnyBurger

Coming in 2023

JonnyBurger avatar Dec 22 '22 10:12 JonnyBurger

Would it be possible to invoke 2 lambda functions at once? One function that just renders the Remotion composition, and the other one is for the overlay. And they would get invoked at the same time so the overlaying function could get the time to load all the media from the s3 bucket, so when the Remotion composition is done rendering, the overlay function already have loaded everything and could basically just start overlaying directly.

Or would this just be unnecessary, or will the overlay and render be on the same lambda function?

Anyways here's a ffmpeg string that I'm using in my application to overlay a mp4 (pretty sure most if not all other video formats work) . ffmpeg -i background.mp4 -i prores-remotion-render.mov -filter_complex '[0:v][1:v]overlay[out]' -map '[out]' -map 0:a -y output.mp4 This works really good and I've had some errors with it, but I'm not thinking that this ffmpeg string causes those problems, but external factors like the VM. Still mostly works fine.

Might give you a bit of a head start at least, then starting from scratch 👍🏻

And also one question, are you planning on early-mid-late 2023?

BenjaminLindberg avatar Dec 22 '22 18:12 BenjaminLindberg

@BenjaminLindberg

Kind of! For overlaying the two tracks, this will for sure happen in the master function. One function for rendering the Remotion part and the background ideally does not need to be preprocessed at all.

Thanks for the script, which is very helpful! I will indeed use it as a reference.

I don't set any targets or deadlines, but it is one of the higher priorities.

JonnyBurger avatar Dec 23 '22 19:12 JonnyBurger

Hello,

Any news for this?

Swarzox avatar Jul 18 '23 09:07 Swarzox