Fragmented videos do not play well on single download

[Tkkg1994]

Describe the current behavior

I came across some issues with the download function on your website. It mostly happens on older uploads and on the new ones it's just fine. The download itself works on all the videos BUT (and this is a big but) it happens that on older uploads, after the download is finished, you start to play to video and you can't navigate in the Video, because there's no time frame in the navigation. You can't skip anywhere and it also doesn't show you how long the video has play / has left to play.

This makes the download function unusable, sadly....

I've tried different browsers for the download, plus jdownloader and even torrent.... Also tried different players such as VLC and the default windows player.

https://tube.kenfm.de/videos/watch/b0f30137-a8f1-4d66-9734-f9472412cedc -> example video which does NOT work https://tube.kenfm.de/videos/watch/32e2055a-6ea7-4e57-86d3-51db52031222 -> example video which WORKS

Steps to reproduce:

1. Go to my linked video from above
2. Click on direct or torrent download (it doesn't matter, I tried both)
3. Wait till the file gets loaded
4. Start the video
5. Then try to move to different times inside the video
6. It won't work on the first video, but it will work on the 2nd video

Describe the expected behavior You can download and watch any Video just fine.

Additional information

• PeerTube instance:

  • URL: https://tube.kenfm.de/
  • version: 3.1.0
  • NodeJS version: can't tell
  • Ffmpeg version: can't tell

• Browser name, version and platforms on which you could reproduce the bug: Windows Edge, Firefox and Opera (Windows 10)

[Chocobozzz]

Videos transcoded with HLS are fragmented videos. Some player have difficulties to play/seek into it.

[boskote]

I'm involved with a peertube instance (https://kolektiva.media) that encountered this problem recently. After upgrading to version 3.1.0 at the beginning of May we followed the recommendations and switched to HLS transcoding. Our storage space is limited so we also disabled the webtorrent transcoding. Immediately we started getting reports of problems, primarily from people using special software to do translation and subtitle work on downloaded files for HLS transcoded videos, but also errors when trying to upload the videos to other social media platforms, as well as buggy seeking/navigation as reported in this issue. Within a week we disabled HLS transcoding and reverted back to webtorrent and this resolved the issues.

In my own testing of fragmented mp4s downloaded from peertube, in addition to timing/seeking issues with VLC and other video players, I have also found that these files are blocked from upload to mastodon by a "File content type is invalid" error. I believe Kolektiva video/subtitle translators were also having additional problems when trying to work with the fragmented mp4s, though I haven't reproduced these myself.

Since then, I have been researching the cause of this problem and possible solutions. I was starting a bug report here when I found this issue, so I'll just comment here instead. I agree that the underlying cause of the problem is that the video files output from the ffmpeg HLS transcoding jobs are fragmented mp4. These file are intended to be streamed using a corresponding .m3u8 playlist file that is also output by the ffmpeg job. Therefore, it is not surprising that there are problems when people are downloading and trying to use the fragmented mp4 as though it is a regular video file: it is primarily intended to be viewed/streamed via its .m3u8 playlist. Treating it as a regular standalone mp4 file (as the peertube download feature implicitly does when only HLS transcoding is enabled) leads to problems for users who aren't getting what they expect when they download a video.

What should be done about this? I think the first thing is to warn administrators about these problems in the documentation. Currently the "VOD transcoding" section of the docs says "We recommand you to enable HLS (and disable WebTorrent if you don't want to store 2 different versions of the same video resolution)" and then "The main drawback is that this HLS with P2P player is not compatible with WebTorrent" and there is nothing mentioned about compatibility issues with downloaded fragmented mp4s. This should be mentioned so that admins can weigh the potential compatibility issues against the streaming advantages (which you do list in the docs), and avoid the experience of Kolektiva of following the recommendations and then reverting because it causes problems for users.

Ideally, there would be a way to fix the fragmented mp4 compatibility problems and allow for HLS-only transcoding as is currently recommended in the docs. This would definitely be our preference at Kolektiva, HLS streaming is clearly much better! I've been doing research and testing about this question, and the best idea I have so far for a solution is that when a user starts a download of a HLS transcoded video, a priority ffmpeg job is initiated that does a remux of the fragmented mp4 to convert it to a raw mp4 (ffmpeg -i fragmented.mp4 -c copy -map 0 regular.mp4). This will increase the time that it takes a user to download the video, but from my tests remuxing is much faster than transcoding so it shouldn't take too long. Presumably the raw mp4 would then be stored on the server, which would be a partial version of the double of storage size that is being avoided by disabling webtorrent transcoding. However I speculate that most interaction with peertube content is through streaming, and downloading videos is much more rare, which means that the increase in storage size will be much less than double. If this were an option for admins, I imagine it being called something like "HLS MP4 download compatibility enabled". I think this kind of feature would make it more practical for Kolektiva to use HLS streaming, and likely the same for lots of other instances as well.

[Tkkg1994]

I'm involved with a peertube instance (https://kolektiva.media) that encountered this problem recently. After upgrading to version 3.1.0 at the beginning of May we followed the recommendations and switched to HLS transcoding. Our storage space is limited so we also disabled the webtorrent transcoding. Immediately we started getting reports of problems, primarily from people using special software to do translation and subtitle work on downloaded files for HLS transcoded videos, but also errors when trying to upload the videos to other social media platforms, as well as buggy seeking/navigation as reported in this issue. Within a week we disabled HLS transcoding and reverted back to webtorrent and this resolved the issues.

In my own testing of fragmented mp4s downloaded from peertube, in addition to timing/seeking issues with VLC and other video players, I have also found that these files are blocked from upload to mastodon by a "File content type is invalid" error. I believe Kolektiva video/subtitle translators were also having additional problems when trying to work with the fragmented mp4s, though I haven't reproduced these myself.

Since then, I have been researching the cause of this problem and possible solutions. I was starting a bug report here when I found this issue, so I'll just comment here instead. I agree that the underlying cause of the problem is that the video files output from the ffmpeg HLS transcoding jobs are fragmented mp4. These file are intended to be streamed using a corresponding .m3u8 playlist file that is also output by the ffmpeg job. Therefore, it is not surprising that there are problems when people are downloading and trying to use the fragmented mp4 as though it is a regular video file: it is primarily intended to be viewed/streamed via its .m3u8 playlist. Treating it as a regular standalone mp4 file (as the peertube download feature implicitly does when only HLS transcoding is enabled) leads to problems for users who aren't getting what they expect when they download a video.

What should be done about this? I think the first thing is to warn administrators about these problems in the documentation. Currently the "VOD transcoding" section of the docs says "We recommand you to enable HLS (and disable WebTorrent if you don't want to store 2 different versions of the same video resolution)" and then "The main drawback is that this HLS with P2P player is not compatible with WebTorrent" and there is nothing mentioned about compatibility issues with downloaded fragmented mp4s. This should be mentioned so that admins can weigh the potential compatibility issues against the streaming advantages (which you do list in the docs), and avoid the experience of Kolektiva of following the recommendations and then reverting because it causes problems for users.

Ideally, there would be a way to fix the fragmented mp4 compatibility problems and allow for HLS-only transcoding as is currently recommended in the docs. This would definitely be our preference at Kolektiva, HLS streaming is clearly much better! I've been doing research and testing about this question, and the best idea I have so far for a solution is that when a user starts a download of a HLS transcoded video, a priority ffmpeg job is initiated that does a remux of the fragmented mp4 to convert it to a raw mp4 (ffmpeg -i fragmented.mp4 -c copy -map 0 regular.mp4). This will increase the time that it takes a user to download the video, but from my tests remuxing is much faster than transcoding so it shouldn't take too long. Presumably the raw mp4 would then be stored on the server, which would be a partial version of the double of storage size that is being avoided by disabling webtorrent transcoding. However I speculate that most interaction with peertube content is through streaming, and downloading videos is much more rare, which means that the increase in storage size will be much less than double. If this were an option for admins, I imagine it being called something like "HLS MP4 download compatibility enabled". I think this kind of feature would make it more practical for Kolektiva to use HLS streaming, and likely the same for lots of other instances as well.

Thank you very much for your detailed answer. This does now make much more sense to me! I agree with you that it would be good to have this described in the documentations plus having an option to enable automatic remux (if admins want this feature).

For now I guess I just download the files and try to manually remux them on my PC, I hope this will solve my issue (at least partially).

Thanks again

Edit: Found a nice google chrome addon, which works pretty well to download videos from peertube. Link to their website: https://www.hlsloader.com/

[kontrollanten]

Thanks for the information @boskote In our case we use a CDN with aggressive caching, so running ffmpeg upon the download request would only affect the server once and then the result is stored in the CDN. But maybe that solution would open up a security issue where a script can bring the server down by calling the download URL for all videos at once? To prevent that case we could make the ffmpeg to run in a high prioritized job, but I guess that can lead to other issues.

[c0xc]

Isn't that an example for why there should be an option to keep the original video file? I think that would make much more sense than now having to run those files through ffmpeg again. To a user, it looks like Peertube breaks the video file and they'd have to fix it manually or find another version elsewhere to save it. That's not a good user experience. There's issue #733 and I opened issue #3980 going in the same direction but with a focus on the id.