Next episode has mismatched language between server and ui

[jmerdich]

Root cause

The autoset options only seem to take effect on non-transcoded media-- the transcode URL is never updated! This means that if the selected audio does not match the default, the audio/subs are in the default but the UI reflects the auto-set variant (showing a language that isn't playing).

Changes

When setting the track in the the auto-set logic, also trivially change the audio index in the transcode URL to match.

This fixes it in most cases, but there may be some edge cases that need to go through the full validation to recreate the URL. I don't know the code well enough for that, though.

Issues

Fixes #3994

[sonarqubecloud[bot]]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
0.0% Duplication

[jellyfin-bot]

This pull request has merge conflicts. Please resolve the conflicts so the PR can be successfully reviewed and merged.

[yodaldevoid]

I can confirm that this PR fixes the issue that I reported in #5168 and the issue reported in #3994.

[jellyfin-bot]

This pull request has merge conflicts. Please resolve the conflicts so the PR can be successfully reviewed and merged.

[yodaldevoid]

I would love to see this merged before the cutoff for Jellyfin 10.9 if at all possible.

[jellyfin-bot]

This pull request has merge conflicts. Please resolve the conflicts so the PR can be successfully reviewed and merged.

[sonarqubecloud[bot]]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.2% Duplication on New Code

See analysis details on SonarCloud

[yodaldevoid]

I attempted to build a Jellyfin container with these changes and found that I needed to rebase on top of master (b53f2d28) in order for the image to build. For whatever reason, these changes don't seem to address the issue after the rebase.

[jmerdich]

@yodaldevoid: as a sanity check: what was the playback method in both the current and 'next' episode (as shown in the playback info under the gear icon)? This fix specifically applies to 'Transcoding' but there may be other edge cases elsewhere.

[yodaldevoid]

First, I can confirm the media was being transcoded as I've been testing with the same media and client this entire time.

Second, I've gone back and double checked my findings and it has only gotten more confusing. When I run jellyfin-web from a development client (e.g. npm start) connected to my stable 10.8.13 backend server this fix works fine. This says to me that there is either something getting messed up by the docker packaging system or there is some incompatibility with non-release versions of the server.

I don't really have the time right now to spin up a second backend server to test this further. If by the time I get around to this again no one else has reproduced this problem, I will look further into it.

[jellyfin-bot]

This pull request has merge conflicts. Please resolve the conflicts so the PR can be successfully reviewed and merged.

[thornbill]

Hey @jmerdich could you fix the merge conflict here? Let's try to get this in since 10.9 is finally out.

[Shadowghost]

Shouldn't this be back ported though?

[thornbill]

I wouldn't be comfortable backporting this without significant testing. iirc this is already supposed to be (partially?) managed server side and I'm just not familiar enough with how it should be working and why it currently does not. 🤷‍♂️

[gurka]

I have this problem as well, on v10.9.2, and I was planning to test this PR. I was thinking that I would first build the official 10.9.2 myself (docker), make sure that the issue is still there, then fix the merge conflict, build and test this PR and see what happens. I haven't looked at the code changes at all and I'm not familiar with the jellyfin code base at all.

But, when trying to reproduce it on the official 10.9.2 I discovered a few things that at least were new to me:

• The issue happens when jellyfin-web starts the next episode and it doesn't matter if I let it automatically start it (at the end of the current episode), or if I click the "Start Now" button that appears when the current episode is about to and or if I simply click the "Skip to next" playback button anywhere in the current episode.
• To fix the issue and make the subtitles appear again you can either switch subtitles to anything except the currently selected one and then back, or you can simply go to another episode and then back. In other words, it always seems to work OK the second time you start an episode.
• Looking at the jellyfin logs while testing this I noticed that, the first time an episode is played by e.g. using "Skip to next", there is no AttachmentExtractor being run. If I then do any of the methods described in the previous bullet point then there is a AttachmentExtractor being run and ffmpeg/ffprobe output logged. Compare the following:

First time an episode is played by e.g. click the "Skip to next" button:

jellyfin-1  | [07:59:14] [INF] [23] MediaBrowser.Controller.MediaEncoding.TranscodingJob: Stopping ffmpeg process with q command for /config/transcodes/a8525c8547a1d47a87f26ffe919fcaa2.m3u8
jellyfin-1  | [07:59:14] [INF] [51] Jellyfin.Api.Helpers.MediaInfoHelper: User policy for simon. EnablePlaybackRemuxing: True EnableVideoPlaybackTranscoding: True EnableAudioPlaybackTranscoding: True
jellyfin-1  | [07:59:14] [INF] [51] Jellyfin.Api.Controllers.DynamicHlsController: Current HLS implementation doesn't support non-keyframe breaks but one is requested, ignoring that request
jellyfin-1  | [07:59:14] [INF] [51] MediaBrowser.MediaEncoding.Transcoding.TranscodeManager: /usr/lib/jellyfin-ffmpeg/ffmpeg -analyzeduration 200M -probesize 1G -f matroska,webm -noautorotate -i file:"/media/[redacted]/S01E15.mkv" -map_metadata -1 -map_chapters -1 -threads 0 -map 0:0 -map 0:1 -map -0:s -codec:v:0 libx264 -preset veryfast -crf 23 -maxrate 6082282 -bufsize 12164564 -x264opts:0 subme=0:me_range=4:rc_lookahead=10:me=dia:no_chroma_me:8x8dct=0:partitions=none -force_key_frames:0 "expr:gte(t,n_forced*3)" -sc_threshold:v:0 0 -vf "setparams=color_primaries=bt709:color_trc=bt709:colorspace=bt709,scale=trunc(min(max(iw\,ih*a)\,min(1920\,1080*a))/2)*2:trunc(min(max(iw/a\,ih)\,min(1920/a\,1080))/2)*2,format=yuv420p" -codec:a:0 libfdk_aac -ac 2 -ab 256000 -ar 48000 -af "volume=2" -copyts -avoid_negative_ts disabled -max_muxing_queue_size 2048 -f hls -max_delay 5000000 -hls_time 3 -hls_segment_type fmp4 -hls_fmp4_init_filename "ea96f95581ab037316d3b69b837c067c-1.mp4" -start_number 0 -hls_segment_filename "/config/transcodes/ea96f95581ab037316d3b69b837c067c%d.mp4" -hls_playlist_type vod -hls_list_size 0 -y "/config/transcodes/ea96f95581ab037316d3b69b837c067c.m3u8"
jellyfin-1  | [07:59:15] [INF] [23] MediaBrowser.MediaEncoding.Transcoding.TranscodeManager: FFmpeg exited with code 0
jellyfin-1  | [07:59:15] [INF] [23] MediaBrowser.MediaEncoding.Transcoding.TranscodeManager: Deleting partial stream file(s) /config/transcodes/a8525c8547a1d47a87f26ffe919fcaa2.m3u8
jellyfin-1  | [07:59:16] [INF] [22] Emby.Server.Implementations.Session.SessionManager: Playback stopped reported by app Jellyfin Web 10.9.2 playing Avsnitt 14. Stopped at 6383 ms

Then, if I just click "Skip to previous" and immediately "Skip to next", to restart this episode 15: (I removed most of the ffmpeg/ffprobe output to not make this comment too long)

jellyfin-1  | [08:02:00] [INF] [15] MediaBrowser.Controller.MediaEncoding.TranscodingJob: Stopping ffmpeg process with q command for /config/transcodes/387153c7da2293bcafce6bcc538d0ce7.m3u8
jellyfin-1  | [08:02:00] [INF] [22] Jellyfin.Api.Helpers.MediaInfoHelper: User policy for simon. EnablePlaybackRemuxing: True EnableVideoPlaybackTranscoding: True EnableAudioPlaybackTranscoding: True
jellyfin-1  | [08:02:00] [INF] [21] Jellyfin.Api.Controllers.DynamicHlsController: Current HLS implementation doesn't support non-keyframe breaks but one is requested, ignoring that request
jellyfin-1  | [08:02:00] [INF] [21] MediaBrowser.MediaEncoding.Attachments.AttachmentExtractor: /usr/lib/jellyfin-ffmpeg/ffmpeg -dump_attachment:t "" -y  -i file:"/media/[redacted]/S01E15.mkv" -t 0 -f null null
jellyfin-1  | ffmpeg version 6.0.1-Jellyfin Copyright (c) 2000-2023 the FFmpeg developers
jellyfin-1  | Input #0, matroska,webm, from 'file:/media/[redacted]/S01E15.mkv':
jellyfin-1  | [08:02:01] [INF] [44] MediaBrowser.MediaEncoding.Attachments.AttachmentExtractor: ffmpeg attachment extraction completed for file:"/media/[redacted]/S01E15.mkv" to /cache/attachments/16e7fda14cfa89e91acc1ec8d5966bb3
jellyfin-1  | [08:02:01] [INF] [44] MediaBrowser.MediaEncoding.Transcoding.TranscodeManager: /usr/lib/jellyfin-ffmpeg/ffmpeg -analyzeduration 200M -probesize 1G -f matroska,webm -noautorotate -i file:"/media/[redacted]/S01E15.mkv" -map_metadata -1 -map_chapters -1 -threads 0 -map 0:0 -map 0:1 -map -0:0 -codec:v:0 libx264 -preset veryfast -crf 23 -maxrate 6082282 -bufsize 12164564 -x264opts:0 subme=0:me_range=4:rc_lookahead=10:me=dia:no_chroma_me:8x8dct=0:partitions=none -force_key_frames:0 "expr:gte(t,n_forced*3)" -sc_threshold:v:0 0 -filter_complex "[0:2]scale=-1:1080:fast_bilinear,scale,crop,pad=max(1920\,iw):max(1080\,ih):(ow-iw)/2:(oh-ih)/2:black@0,crop=1920:1080[sub];[0:0]setparams=color_primaries=bt709:color_trc=bt709:colorspace=bt709,scale=trunc(min(max(iw\,ih*a)\,min(1920\,1080*a))/2)*2:trunc(min(max(iw/a\,ih)\,min(1920/a\,1080))/2)*2,format=yuv420p[main];[main][sub]overlay=eof_action=pass:repeatlast=0" -start_at_zero -codec:a:0 libfdk_aac -ac 2 -ab 256000 -ar 48000 -af "volume=2" -copyts -avoid_negative_ts disabled -max_muxing_queue_size 2048 -f hls -max_delay 5000000 -hls_time 3 -hls_segment_type fmp4 -hls_fmp4_init_filename "dfe81db7edf9b0ebe95deef2b511dc8a-1.mp4" -start_number 0 -hls_segment_filename "/config/transcodes/dfe81db7edf9b0ebe95deef2b511dc8a%d.mp4" -hls_playlist_type vod -hls_list_size 0 -y "/config/transcodes/dfe81db7edf9b0ebe95deef2b511dc8a.m3u8"
jellyfin-1  | [08:02:01] [INF] [15] MediaBrowser.MediaEncoding.Transcoding.TranscodeManager: FFmpeg exited with code 0
jellyfin-1  | [08:02:01] [INF] [15] MediaBrowser.MediaEncoding.Transcoding.TranscodeManager: Deleting partial stream file(s) /config/transcodes/387153c7da2293bcafce6bcc538d0ce7.m3u8
jellyfin-1  | [08:02:02] [INF] [15] Emby.Server.Implementations.Session.SessionManager: Playback stopped reported by app Jellyfin Web 10.9.2 playing Avsnitt 14. Stopped at 4708 ms

I'm not sure if this helps in any way. It would be interesting to see exactly the requests that jellyfin-web sends to jellyfin in these two scenarios, but I'm not sure what kind of logging I need to enable and how to do that.

I haven't tested this PR yet, I will try to do that later today and report back.

[gurka]

I've tested this PR now and it works OK. I've tested skipping to next episode, waiting for the end of an episode and clicking "Start now" and just waiting for the current episode to end and let it auto-play the next episode, and every time the previously selected subtitles are displayed correctly.

However, at some point I noticed that my server which is hosting jellyfin was using 100% CPU even though I were no longer streaming anything. It looked like there were many jellyfin ffmpeg processes running. Probably not related to this PR, maybe there is another issue for that. Or I was just skipping episodes too quickly :D

I will continue to use this PR on my server and will report back if I find any other issues or scenarios where it does not work correctly. Note that I've only tested the subtitles aspect of this functionality, not audio tracks.

[ebkalderon]

Would it be possible to fix the merge conflicts on this PR and re-request reviews, please? I would love to see this long-awaited PR land in time for the 10.9.10 or 10.10.0 release. This is quite a long-standing bug preventing subtitles from working properly on transcoded media, and it's the biggest source of everyday complaints from family and friends using my home Jellyfin instance. I'd love to see this issue finally put to rest very soon. Thanks again for the lovely PR. :heart:

[gurka]

I can just add that I've been using this patch for about 3 months without any issues (neither the issue described in the ticket nor any other issue).