Subtitles from external file

[philstopford]

In the GUI 'streams' panel, I see two tabs, one for audio and one for subtitles. Switching to subtitle tab, the 'add external stream' button does not allow me to add an external subtitle file (.srt, for example). It seems that the tool currently only supports audio for external streams.

Would it be possible to allow for external subtitles to be added? If so, having the option to either retain them as a separate stream in the output, or to burn them into the video would be very welcome.

For the burning into video, it would be helpful to have a preview option so that the burn-in could be configured according to the user's needs (e.g. to avoid white text over white in the frame, having a background translucent dark box is often helpful)

[CineEncoder]

Hello! Adding external subtitles is easy to do, and previewing subtitles is much more difficult. If you mean a preview of the original video, then that's another matter.

[philstopford]

My thought had been that, for preview, one could perhaps provide a start and end time, with the tool then calling ffmpeg behind the scenes to generate an output file that could be passed to VLC or similar. I would guess this goes with a general preview implementation, which I think I saw discussed here already. It's not unique to subtitles, but being able to view the output of hard-burned subtitles before processing the entire video would be neat.

[CineEncoder]

Hello! Now there is already a tool for previewing the source video for cutting (View->Split), perhaps it will be possible to modify it somehow.

[CineEncoder]

Hello! I added on the branch 'develop' support for external subtitles, and the ability to set a default track for audio and subtitles, but I haven't checked it properly yet. https://github.com/CineEncoder/cine-encoder/tree/develop

[philstopford]

I took this for a spin and it looked like a subtitle stream was present in the mp4 output, but was empty (no subtitles appeared in the video when activated through VLC). Now, looking at the preset, I don't see a subtitle output section, so wonder if the output is losing the stream contents.

[CineEncoder]

Hello! I got the following result: if the external subtitles were obtained from the source file, then they are played correctly, if not, they are simply displayed in the list but are not played. Perhaps as in the case of audio, they also need to synchronize their duration with the video file. Tried for containers mp4, mov, mkv. The log does not show the paths of external files, but they are present. The preset for a video file without audio and with one external subtitle looks like this for mkv: -hide_banner -hwaccel cuda -i <input file> -i <extern_subtitle> -map 0:v:0? -map 1:s? -map_metadata -1 -map_metadata:s:v:0 -1 -metadata:s:s:0 language=en -metadata:s:s:0 title=External -disposition:s:0 default -pix_fmt p010le -c:v hevc_nvenc -profile:v main10 -preset slow -b:v 50000000 -minrate 50000000 -maxrate 50000000 -bufsize 50000000 -rc vbr -2pass 1 -color_range tv -color_primaries bt2020 -colorspace bt2020nc -color_trc smpte2084 -c:a ac3 -b:a 448k -c:s copy -y <output file>

[philstopford]

Syntax reference from a comparable tool, for what it is worth

Soft-burn from external subtitle file: "ffmpeg.exe" -probesize 50M -analyzeduration 100M -i "Z:\T7AW5W~N.MP4" -sub_charenc UTF-8 -i "Z:\T7AW5W~N.SRT" -map 0:0 -map 0:1 -map 1:0 -c:a aac -ab 128k -strict -2 -async 1 -c:v libx265 -r 24000/1001 -s 1920x1080 -aspect 16:9 -pix_fmt yuv420p -c:s mov_text -metadata:s:s:0 language=eng -preset fast -x265-params crf=25:me=hex:qcomp=0.5:scenecut=40:bframes=4:min-keyint=24:keyint=240 -qmin 3 -qmax 51 -qdiff 4 -tag:v hvc1 -metadata creation_time=now -y "Z:\TGPKAA~W.MP4"

Hard-burn from external subtitle file: "ffmpeg.exe" -probesize 50M -analyzeduration 100M -i "Z:\T7AW5W~N.MP4" -map 0:0 -map 0:1 -c:a aac -ab 128k -strict -2 -async 1 -c:v libx265 -r 24000/1001 -s 1920x1080 -aspect 16:9 -pix_fmt yuv420p -preset fast -x265-params crf=25:me=hex:qcomp=0.5:scenecut=40:bframes=4:min-keyint=24:keyint=240 -qmin 3 -qmax 51 -qdiff 4 -tag:v hvc1 -metadata creation_time=now -vf "subtitles='Z\:\\T7AW5W~N.SRT'":charenc=utf-8 -sn -y "Z:\TGPKAA~W.MP4"

With background and stuff, for hard-burn: "ffmpeg.exe" -probesize 50M -analyzeduration 100M -i "Z:\T7AW5W~N.MP4" -map 0:0 -map 0:1 -c:a aac -ab 128k -strict -2 -async 1 -c:v libx265 -r 24000/1001 -s 1920x1080 -aspect 16:9 -pix_fmt yuv420p -preset fast -x265-params crf=25:me=hex:qcomp=0.5:scenecut=40:bframes=4:min-keyint=24:keyint=240 -qmin 3 -qmax 51 -qdiff 4 -tag:v hvc1 -metadata creation_time=now -vf "subtitles='Z\:\\T7AW5W~N.SRT'":charenc=utf-8:force_style="'FontName=Arial,Fontsize=20,PrimaryColour=&H00FFFFFF,BorderStyle=4,BackColour=&H96000000'" -sn -y "Z:\TGPKAA~W.MP4"

[CineEncoder]

Good day! It seems to be working better now, I found an error, I set the order of the external tracks incorrectly. Here is the corrected version: https://github.com/CineEncoder/cine-encoder/tree/prototype/file_browser (don't pay attention to the 'Cut by shortest' button, it don't work yet)

[philstopford]

Indeed - that works well for the soft subtitle option. Thanks!

[philstopford]

Do you think it might be possible to add hard-burn in the code? It's going to be a few weeks until I get myself set up to dig into the project.

[CineEncoder]

Hello! I'll have to see, I still don't fully understand the difference between soft and hard-burn.

[philstopford]

So hard-burn bakes the subtitles into each frame, so there's no additional stream in the output. There's some related overview here : https://handbrake.fr/docs/en/1.5.0/advanced/subtitles.html

The desire to hard-burn subtitles can come from video format or playback system limitations, or just for convenience, or because the user has certain formatting preferences (font preferences, ability to set a background to allow the light subtitle text to be read over a bright background in the frame).

[CineEncoder]

Hello! I think it can be done.

[philstopford]

So now that I have the code building, I can see that the column headers appear to be shared between audio and subtitles, so placing the appearance controls into the table would make it unwieldy and also clutter the audio view from what I can tell.

I'm thus scratching my head as to how the GUI could support the hard burn side of things, if only a checkbox or similar could be placed into the table layout (similar to the default radio button for audio encoding where multiple tracks are present)

I thought about storing the appearance values in preferences, but no other part of the encode information seems to be stored there. Presets are 'volatile' in that they get reset each update, so may not be a good place to store this, either. I'm pretty much a novice with Qt code, and only lightly familiar with C++ (being more of a C# person), so haven't started writing code yet - just trying to figure the existing code out :)

[CineEncoder]

Yes, I roughly know how to make this option with subtitles, I'll add it later, there is a rather complicated implementation. Data for each track of audio and subtitles is stored here, for each video file, depending on the index of the video file in the table. https://github.com/CineEncoder/cine-encoder/blob/master/app/mainwindow.h#L169 https://github.com/CineEncoder/cine-encoder/blob/master/app/constants.h#L101

[philstopford]

I took a look at this today and the UI implementation puzzles me. It looks like the tables are shared between the audio and subtitle features. If I add a 'burn' option, the UI elements seem to get drawn for both. I didn't see an obvious way to hide the burn option from the audio table presentation.

[CineEncoder]

I make this option on this branch: https://github.com/CineEncoder/cine-encoder/commit/b59d8f17fc9d1f949a0cad4da4fced8d8f417e37

here the radio button is displayed only for subtitles, it looks like this:

Here, the various combinations have yet to be properly processed so that the remaining buttons switch correctly when this option is enabled or disabled.

[philstopford]

Neat - thanks for this. I was trying to figure out where to set up the font, etc. settings for this. I'm actually tempted to think about a global preference to make it easy, unless you want to host it in the presets instead.

[CineEncoder]

It seems to me that the font settings for embedded subtitles can indeed be made global, they could be placed in these tabs, after the Audio section. I'm thinking of moving support for embedded subtitles and the ability to preview videos to the next releases? because there is still a lot of work to be done.

[philstopford]

Yes - I was looking at the existing code for subtitle handling and it has me slightly puzzled:

    /************************************ Subtitle module *************************************/

    QString sub_param("");
    if (container == "mkv") {
        _sub_mux_param = QString("-c:s ass");
    }
    else
    if (container == "webm") {
        _sub_mux_param = QString("-c:s webvtt");
    }
    else
    if (container == "mp4" || container == "mov") {
        _sub_mux_param = QString("-c:s mov_text");
    }
    else {
        _sub_mux_param = QString("-sn");
        emit onEncodingError(tr("Container \'%1\' will be transcoded without subtitles.")
                             .arg(container), true);
    }

    if (_flag_hdr) {
        sub_param = QString(" -c:s ass");
    }
    else {
        sub_param = QString(" ") + _sub_mux_param;
    }

I haven't spent enough time with the code yet to see how to capture that the user wants to hard burn vs copy.

I need to do some reading, but it surprised me that the nature of the subtitle was forced by format (e.g. ass for mkv, mov_text for mp4), and that the HDR flag forces ass no matter what the previous conditionals set-up - looking at this code block, a switch statement might also be clearer (you have quite a few similar cases of many if/else if statements that might be cleaner as switch statements). I had in mind to look at that first, to try and simplify the code.

In any case, for hard burning, we'd need to have a new video filter module, I think, that would allow for the hard burning to be encoded (this would also be extensible in future to additional filters and perhaps crop/resize).

At that point, preview becomes very important so I think the two things have to end up in the same release, otherwise the user will be frustrated.

[CineEncoder]

Yes, the subtitle format depends on the selected container. As for the code:

if (_flag_hdr) {
    sub_param = QString(" -c:s ass");
}
else {
    sub_param = QString(" ") + _sub_mux_param;
}

here sub_param is used for intermediate encoding to .mkv format if _flag_hdr is true. Intermediate conversion to .mkv format is used for correct transfer of HDR parameters.

[philstopford]

See also https://github.com/CineEncoder/cine-encoder/pull/70