spacechop
Video Support
@argenisleon do you mean to support transforming encoded video streams or rather to generate screenshots from videos or both?
Closest in range regarding video I would say is to support transforming animated webp and gif, it would be a larger effort to support video.
@timbrandin I mean generate screenshots from video. Maybe gif sequences too?
@timbrandin I mean generate screenshots from video. Maybe gif sequences too?
Alright, that would be interesting! To support video as input, does one want to specify at what time in the video or just the first frame?
What do you think about the first frame of the video and random sections of the video to generate a gif preview?
Yeah, that would be a great start I guess.
What video formats would be a good start?
Also how would one fetch the source, in most cases with pictures people store them on s3 or similar object storages, what would be the preference for video?
Also what use case would this serve?, that i.e. would be great to know to understand and prioritize tasks for this.
It would be .mp4, could be fetch from url. This would serve to create a thumbnail and gif from a video to display the preview from said video.
It would be .mp4, could be fetch from url. This would serve to create a thumbnail and gif from a video to display the preview from said video.
Cool, we support streaming from http sources already. I'll investigate how complex this would be and report my findings here.
Most likely we can solve this using ffmpeg.
And in my research I also stumbled upon nginx capabilities of working as a pseudo streaming server: http://nginx.org/en/docs/http/ngx_http_mp4_module.html If one is able to setup this on the backend hosting the video files, that could improve transformation speed as one does not have to download the full video, and rather seek to a specific position.
Btw, @EdoGp @argenisleon how big are your video files in average?
Max 50mb video files, I tried using video-thumbnail-generator . that uses ffmpeg, but it uses to many resources from the server, that is why I'm looking for an alternative
@EdoGp Alright, that's a good benchmark then. But what resources are we talking about; memory, disk or cpu? And just to dig into why it's using so much resources:
- was
video-thumbnail-generatorrunning on the same machine hosting the files or fetching them over the network? - Did you use
video-thumbnail-generatorto generate thumbnails or gifs or both?
I've tried it a bit now, was pretty easy to setup, though it seams to just wrap fluent-ffmpeg which might be doing more than needed for just generation of thumbnails. Also it was pretty difficult to get out a screenshot from the video with the original aspect ratio intact. I might be missing some setting, but the default gave me a square in which the video frame was compressed into. I'm guessing one would rather want to crop and set gravity center instead or maybe add a blur or something behind.
I'm gonna look into ffmpeg to start with, though I need some more info on where you hit the limit on the resources to get a really good benchmark to aim for, if you don't mind giving me some more info regarding this. 💃
1- Yes, video-thumnail-generator was running on the same machine as the files.
2- Yes, video-thumnail-generator was being used to generate both, thumbnails and gifs
The problem came when we wanted to process several videos simultaneously, the processor use came to a 90% high with 3 videos processing at the same time, and we need it to be able to scale more effectively.
@EdoGp alright, then I'm guessing they were loading all the bytes into memory before generating thumbnails, I'm also guessing one might need a larger machine to run this on as this will be quite memory consuming if not optimized.
Maybe we can find a way to optimize this, my first though is to try moving all byte handling of the javascript thread, which is kind of how spacechop works at the moment so hopefully this could get better performance.
Another idea is to copy how nginx does pseudo streaming when streaming from a local volume, this could hopefully reduce the need to load all of the bytes into memory.
I'll investigate some more next week, feel free to look into how spacechop works in the meantime, a good start is index.ts in src/.
As a note: regarding mp4 to animated gif I found this: https://www.imagemagick.org/discourse-server/viewtopic.php?t=34512
so there could be a better way to generate the preview image of the video with spacechop? I'll look into spacechop src/