ShaneIsrael
Add HLS support
To help mitigate bandwidth limitations and overall provide a better video viewing experience, this PR adds transcoding video + audio to m3u8, and adds the /api/video/stream endpoint to return it.
I've also updated the video-players used on the site to handle this - Previously the VideoModal used the native video element, it now uses ReactPlayer, like the Watch page.
There's a chance it may fix discord embeds, will check soon
Todo before ready to be merged:
- [ ] Default to direct streaming unless admin has changed default mode to transcode via toggle on settings page
- [ ] Setting for enabling or disabling transcoding totally, disable admin option for default video play mode
- [ ] Setting on video players for playback mode?
- [ ] New video uploaded and playback started before transcode done
- [ ] Show video processing page
- [ ] Show original video instead
- [ ] Have an option of this behaviour?
- [ ] Properly test playback on Discord and different devices
- [ ] Option for quality of playback and a default setting? - I like having my clips high res + bitrate if I were to turn it into a video, but for little clips to share with friends it doesn't need much quality
- [ ] Original (still HLS)
- [ ] Low (720p30)
- [ ] Medium (1080p60)
- [ ] Convert beforehand or at start of request? - on-the-go (see below) is probably the only feasible option
- [ ] Look into transcoding each segment of the video on-the-go, for proper transcoding, higher performance requirements but may reduce overall disk usage especially if having multiple quality options
- [ ] Check if ReactPlayer has options for a YouTube-like quality selector beforehand
My only concern with this is defaulting the videos to the m3u8 conversion. If you add or upload a video, the video player would fail to load the video if the conversion is still happening. The nice thing about direct file stream is that the video is ready immediately, this is especially important for systems that do not have much processing power and may otherwise take a while to convert to m3u8.
I think what I would need to have added for this PR to go through are a couple of things.
- The video player should default to direct streaming of the original file unless the administrator has changed the default streaming mode from "direct" to "transcode".
- For the above to work, there should be a new setting in the frontend ui that allows the admin to set the default streaming mode. "direct" or "transcode".
- Another setting needs to also be added to "enable / disable" video transcoding. There are some users who are not going to want their videos transcoded because it would impact their systems processing too much and they may prefer to only direct stream files.
- When visiting a video that isn't yet available because it is still transcoding, i.e the m3u8 file is not available and "direct" stream is not the default mode, there page should show some form of "video is processing" component in place of the video player. Or, potentially the video should load the direct stream version first while the video is still being transcoded.
Some of these settings could probably be added as .env variables, and may actually be more appropriate there. Of course I'm open to suggestions on these points. What I want to make sure of is that adding this feature, doesn't impact those who are relying on direct stream because they are on a low power system or maybe they have the bandwidth and would prefer not to use of their cpu transcoding.
One last thought, how is this going to handle existing files? Just default to direct stream is there isn't an m3u8 file for older videos? Or a cron job to convert older videos?
Yeah I was thinking of potentially having some options for this behaviour too, new features aren't exactly great if they hinder the experience of other users. All your suggestions seem great, I'll look into adding them.
Conversion is handled alongside poster generation so old videos should be upgraded with a scan. In the scenario that an old video is attempted to be viewed before a scan is run, I'll just show the processing page or default to the original file.
For conversion, it's just copying the existing (containers?) to a m3u8 format so it should be pretty quick (0.2 seconds on an i5-8400 for a 10 second, 1440p60 50mbps clip), but still I understand about some users not wanting this behaviour or wanting to sacrifice the performance for it (and it'd probably still take a lot longer on a 10GB+ video anyway)
Works with discord embeds (Didn't work for 10 sec or so, and then it worked so potentially discord needs to scan it first before it's fully available), tested with a 10 second, 65MB file, on an account without any nitro.
Edit: Looks like it was still using the original video, trying again with it actually using m3u8
Yeah I was thinking of potentially having some options for this behaviour too, new features aren't exactly great if they hinder the experience of other users. All your suggestions seem great, I'll look into adding them.
Conversion is handled alongside poster generation so old videos should be upgraded with a scan. In the scenario that an old video is attempted to be viewed before a scan is run, I'll just show the processing page or default to the original file.
For conversion, it's just copying the existing (containers?) to a m3u8 format so it should be pretty quick (0.2 seconds on an i5-8400 for a 10 second, 1440p60 50mbps clip), but still I understand about some users not wanting this behaviour or wanting to sacrifice the performance for it (and it'd probably still take a lot longer on a 10GB+ video anyway)
Yeah I would assume on most systems and videos it should be fine. But you never know how large a video somebody might have or what their circumstances could be. So I'd like to be pretty careful with that. I also think having both options provides a backup in the event that the transcode is not working right or a bug gets introduced. There's at least a fallback option.
Either way, I'm super looking forward to this. Thank you very much for the work put in, sorry if I seem to be making this difficult haha.
Works with discord embeds (Didn't work for 10 sec or so, and then it worked so potentially discord needs to scan it first before it's fully available), tested with a 10 second, 65MB file, on an account without any nitro.
Interesting, because I believe discord embeds have always worked for me without issue. Here is a screenshot of me testing it right now, maybe try yourself linking one of the videos on https://v.fireshare.net
I think the issue is for people who are not encoding their videos in h264, and maybe using h265 which I imagine discord doesn't support.
No problem, I completely get it. I've been using fireshare on and off (linked to how inconsistently I get into games) for about 1.5 years now, so I'm happy to contribute something to it :)
Interesting, because I believe discord embeds have always worked for me without issue. Here is a screenshot of me testing it right now, maybe try yourself linking one of the videos on https://v.fireshare.net
I think the issue is for people who are not encoding their videos in h264, and maybe using h265 which I imagine discord doesn't support.
I've had issues with it before,I believe it was if the video was larger than nitro would usually let you upload the video directly, but apparently not. Will see if this improves the experience at all anyway
Interesting, because I believe discord embeds have always worked for me without issue. Here is a screenshot of me testing it right now, maybe try yourself linking one of the videos on https://v.fireshare.net I think the issue is for people who are not encoding their videos in h264, and maybe using h265 which I imagine discord doesn't support.
I've had issues with it before,I believe it was if the video was larger than nitro would usually let you upload the video directly, but apparently not. Will see if this improves the experience at all anyway
Yep I am all for improved experiences. I know I've seen some people complain about discord embeds not working which has been difficult for me to nail down because they've always just worked for me. If this helps with that then great!
I've added a TODO list to my original message, please let me know what you think of it and any additions I've made to the list
Default to direct streaming unless admin has changed default mode to transcode via toggle on settings page
This is the default behavior I would prefer. 👍 (potentially 3 options if and only if you wanted to add it. The third option could be "auto" or something along those lines which might enable the user the ability to select on the video player?) That could always be something looked into after the fact at a later date.
* [ ] Setting for enabling or disabling transcoding totally, disable admin option for default video play mode
Yep, this would probably be an env variable that is "true" by default. ENABLE_HLS_TRANSCODING=true if false, disable the admin options in the settings UI. A tooltip if its disabled to notify the admin that they need to enable the env variable to allow for transcoding.
* [ ] Setting on video players for playback mode?
I'll leave this up to your own discretion. If its simple enough to add and you feel like it then I think allowing the end user the option to choose if they want to watch as direct stream vs transcode would be nice. But I think this option should only be enabled if an admin setting enables it via that 3rd option I mentioned above. "Direct Stream, Transcode, User Decides" etc or something like that. User Decides probably isn't the best name for it but its early morning and I'm still drinking my coffee haha.
* [ ] New video uploaded and playback started before transcode done * [ ] Show video processing page * [ ] Show original video instead * [ ] Have an option of this behaviour?
A default behavior that the admin can set in the settings is probably ideal but I don't expect that to be required for this PR. Without the option, I think "direct stream" should be the default until the transcode file is available, then default to whatever the admin has set.
* [ ] Properly test playback on Discord and different devices
👍
* [ ] Option for quality of playback and a default setting? - I like having my clips high res + bitrate if I were to turn it into a video, but for little clips to share with friends it doesn't need much quality * [ ] Original (still HLS) * [ ] Low (720p30) * [ ] Medium (1080p60) * [ ] Convert beforehand or at start of request? - on-the-go (see below) is probably the only feasible option
Hmm. I think there should be a default settings, something to high res + bitrate because like you I also prefer quality. I do think 3 options should be available, but maybe that should be a setting the admin can set because it might be rough on the system having it transcode a video into 3 different files.
So what do you think about these for options?
- Default - Original (still HLS)
- High - high quality, high res + bitrate
- Medium
- Low
- ALL
Where option 5 all would tell the transcode utility to create a file for each quality profile. The user could then select the quality profile if available on the video player?
GPU transcoding would greatly help here. I'm not sure how complicated it would be to pass in your nvidia/amd gpu to the container to be used by ffmpeg. I was at one point experimenting with nvidia nvenc encoder with ffmpeg but didn't make too much progress on it. Thoughts?
* [ ] Look into transcoding each segment of the video on-the-go, for proper transcoding, higher performance requirements but may reduce overall disk usage especially if having multiple quality options
Feel free to experiment with this, but definitely not required. Would probably need gpu transcoding to properly handle this.
* [ ] Check if ReactPlayer has options for a YouTube-like quality selector beforehand
Definitely think this would be cool especially if we have 4 different quality files created for each video.
Hi, I'm still looking to add this. I've played around with transcoding in realtime for different qualities, I'm not sure if I'll be able to add it due to time and knowledge limits. If you or someone else would like to work on it later, a decent start has been made on it. Alternatively the different quality versions of the videos could be created beforehand, but this would obviously use more space and processing power when uploading the videos.
I'll do the other things in my list, hopefully within the month 👍
Hi, I'm still looking to add this. I've played around with transcoding in realtime for different qualities, I'm not sure if I'll be able to add it due to time and knowledge limits. If you or someone else would like to work on it later, a decent start has been made on it. Alternatively the different quality versions of the videos could be created beforehand, but this would obviously use more space and processing power when uploading the videos.
I'll do the other things in my list, hopefully within the month 👍
Sure no problem! The project isn't going anywhere, there's no expectation or time limits for open source contributions :)