Fixing Live TV since the refactor

[jhays]

I'm trying to repair LiveTV functionality, which has been in a broken state since the last big refactor work / video player update on this project.

This is by no means complete code or ready to merge, so I'm raising this draft PR with the hopes of generating some discussion and getting feedback on how I should go about fixing these issues. In the short term, the changes in this PR at least get me back to a state where I am able to play back Live TV content (though without any overlay functionality at the moment).

I'm having some trouble wrapping my head around some of the new player refactor work, and where certain environmentObjects are intended to be getting set. I'm happy to put further work into the Live TV feature, but I'm hoping I can get some feedback on what is needed to get LiveTV working well with the new player architecture since the refactor.

I'll mark the issues I've run into in comments on the changes.

[jhays]

Hey @LePips 👋

If you have some time to check this out, it would be great to get some direction from you on what you might be envisioning in order to get LiveTV implemented in a way that works well with the new player architecture.

[sy6sy2]

Hey @jhays, thank you for your work! I just tried your branch with latest commits, unfortunately my app stuck in the Retrieving media information wheel when I try to play a TVheadend channel.

Xcode log:

NavigationLink presenting a value must appear inside a NavigationContent-based NavigationView. Link will be disabled.
NavigationLink presenting a value must appear inside a NavigationContent-based NavigationView. Link will be disabled.
didset orientations: UIInterfaceOrientationMask(rawValue: 26)

Jellyfin log:

mediaserver-jellyfin-1  | [14:47:30] [INF] [65] TVHeadEnd.AccessTicketHandler: [TVHclient] AccessTicketHandler.GetAccessTicket: New ticket (#2) created for channelId=134683571
mediaserver-jellyfin-1  | [14:47:30] [INF] [65] Jellyfin.Api.Helpers.MediaInfoHelper: User policy for sylvain. EnablePlaybackRemuxing: True EnableVideoPlaybackTranscoding: False EnableAudioPlaybackTranscoding: False
mediaserver-jellyfin-1  | [14:47:30] [INF] [65] Jellyfin.Api.Helpers.MediaInfoHelper: StreamBuilder.BuildVideoItem( Profile=Anonymous Profile, Path=http://192.168.1.20:9981/stream/channelid/134683571?ticket=d66e0c293bb50c90387c5f0c8b18378960bcab46, AudioStreamIndex=null, SubtitleStreamIndex=null ) => ( PlayMethod=Transcode, TranscodeReason=ContainerNotSupported, VideoCodecNotSupported, AudioCodecNotSupported ) media:/videos/98b482de-5d0f-9bae-fe0d-ac6f60a7d015/master.m3u8?MediaSourceId=134683571&VideoCodec=hevc,h264&AudioCodec=aac,mp3,ac3,eac3,flac,opus&AudioStreamIndex=-1&VideoBitrate=359360000&AudioBitrate=640000&api_key=<token>&TranscodingMaxAudioChannels=6&RequireAvc=false&SegmentContainer=mp4&MinSegments=2&BreakOnNonKeyFrames=True&TranscodeReasons=ContainerNotSupported,%20VideoCodecNotSupported,%20AudioCodecNotSupported

It works well on macOS Jellyfin player with direct play.

[jhays]

Hey @jhays, thank you for your work! I just tried your branch with latest commits, unfortunately my app stuck in the Retrieving media information wheel when I try to play a TVheadend channel.

@sy6sy2 ,Thanks for letting me know. I see the same line in my Xcode log when loading the player for a liveTV channel on iOS, I don't think it is necessarily the problem. But playback is working fine for me on both tvOS and iOS. I'm a little confused here, because to my knowledge, there is no macOS target in this project. At least I don't see one in the codebase I am working on... so I'm not sure what you are running.

At the moment, I'm not sure what issue you are running into, but my first guess is that the logic I added to BaseItemDto+VideoPlayerViewModel.swift to do the media source matching might be a bit brittle, and perhaps that logic is not working with your setup. When it tries to load the player view, but no matching media source is found, perhaps that could explain what you are seeing.

I left a comment in this PR regarding that change. I'm a little unsure on how that media source matching should be implemented with these live tv channels. I'm hoping someone with more Jellyfin API knowledge could weigh in on it.

[sy6sy2]

I'm a little confused here, because to my knowledge, there is no macOS target in this project. At least I don't see one in the codebase I am working on... so I'm not sure what you are running.

Sorry for the confusion, I just wanted to confirm that the problem does not come from my Jellyfin server or my TVHeadend server because Live TV feature works well with the official Jellyfin Media Player for macOS from here 😉

I will check your comment, thanks!

[jhays]

@sy6sy2 thank you for clarifying. I actually didn't know there was a different macOS client!

[jhays]

I am continuing to work on this a bit. I've noticed that with Force Direct Play on, I can play back my ErsatzTV based channels just fine, but my IPTV channels don't work. I've dug into it a bit and noticed that I am getting different responses from the server in these cases. I'm not sure what the deal is, so I've raised an issue here to see if I can get further help:

https://github.com/jellyfin/jellyfin/issues/10168

And then with Force Direct Play OFF, I've run into a transcoding issue where FFmpeg bombs on my IPTV channels due to some aac audio issue. But the same channel plays fine in Jellyfin web browser, or in VLC directly. I've raised that issue here:

https://github.com/jellyfin/jellyfin/issues/10167

Hopefully we can get to the bottom of those two issues and get playback working across more Live TV cases / configs.

[jhays]

Latest commit fixes some Live overlay separation for iOS and adds a slight refresh to iOS Live TV Channels UI. Also re-implemented the Live Channels guide refresh logic.

I think I'm ready to move forward with these changes, so I will be removing the Draft label from the PR. I still have a bit better compatibility with the Native player, but will continue investigating the issues I was seeing with the Swiftfin player.

[LePips]

I've started to take a look at this, it'll just take a bit for me to play around with.

[jhays]

I've started to take a look at this, it'll just take a bit for me to play around with.

👍 let me know if you have any questions or feedback!

[jhays]

I'm running into an issue building this branch on Xcode 15 for iOS/tvOS 17, and I'm having trouble understanding what's going on.

It seems to error during build when attempting to build the target for the dependency CoreStore. I don't think I see this same issue on the main branch, yet I don't think there should be any dependency differences between these branches. At the moment, I'm not sure what's going on.

[jhays]

lol this is so frustrating...

I can't build this branch in Xcode 15 due to it throwing compilation errors within the CoreStore framework, largely in DiffableDataSource and DynamicObject files.

I've tried both rebasing or merging with latest from main, but I run into the same issue.

If reset all the work I've done in this branch, and save it as a stash in git, then create a new branch off main, apply the stash, then everything is fine. Project builds and my changes work.

If I do the above, copy the entire project file structure, and then paste it over this branch after it has been rebased or merged with main, I still end up running into the same CoreStore issues and cannot build. What?!? Is there some untracked file that sets dependency version numbers or something?

EDIT: See below, just needed to reset package cache.

[LePips]

CoreStore just needs to be updated to 9.1.0 which was done here and make sure to reset your package cache. Your branch has the old .resolved.

[jhays]

hah, ok reset package cache was all I needed, thank you. I had been doing "Resolve Package Versions" when what I needed was to reset the cache.

This branch is back to building fine for me in Xcode 15.

[sy6sy2]

Hi @jhays, it's been a while since my last test of your branch but it's seems that Live TV channels coming from TVHeadend plugin don't work anymore 😕

When selecting a channel the video player appears but I get stuck with the spinning wheel (with Jellyfin and Native players)

[jhays]

Hi @jhays, it's been a while since my last test of your branch but it's seems that Live TV channels coming from TVHeadend plugin don't work anymore 😕

When selecting a channel the video player appears but I get stuck with the spinning wheel (with Jellyfin and Native players)

Ah shoot- in this recent test, are you trying on tvOS or iOS?

Have you tried toggling the Experimental > Live TV > Live TV Force Direct Play option?

Ultimately there are 4 player permutations to try:

• Swiftfin (VLCKit) player + Live TV Force Direct Play Enabled
• Swiftfin (VLCKit) player + Live TV Force Direct Play Disabled
• Native Player + Live TV Force Direct Play Enabled
• Native Player + Live TV Force Direct Play Disabled

Please let me know if any of those paths work at all for you, or if it is stuck regardless.

[sy6sy2]

I try on tvOS.

This is my test results:

• Swiftfin (VLCKit) player + Live TV Force Direct Play Enabled --> Black screen
• Swiftfin (VLCKit) player + Live TV Force Direct Play Disabled --> No audio but video works with a lot of "buffering" issues
• Native Player + Live TV Force Direct Play Enabled --> Black screen with spinning wheel
• Native Player + Live TV Force Direct Play Disabled --> Black screen with spinning wheel

Maybe I'm wrong but I'm pretty sure that Live TV worked without transcoding before.

[jhays]

Thanks for the info. I'll have to do some further digging, or maybe add some additional logging to help us see what's actually going wrong. If you are running from Xcode debugger, if you can copy the logs from when you experience the issue and send them to me, I can look and see if anything stands out.

[sy6sy2]

Thank you!

This is the logs with iOS + VLCKit player + force direct play:

Jellyfin server:

mediaserver-jellyfin-1  | [01:02:16] [WRN] [40] Jellyfin.Server.Middleware.ResponseTimeMiddleware: Slow HTTP Response from http://192.168.1.2:8096/Items/Filters2?userId=fdb0bde0fb454866a30d45dd877aa48e to 192.168.1.82 in 0:00:00.6741992 with Status Code 200
mediaserver-jellyfin-1  | [01:02:16] [WRN] [80] Jellyfin.Server.Middleware.ResponseTimeMiddleware: Slow HTTP Response from http://192.168.1.2:8096/Items?userId=fdb0bde0fb454866a30d45dd877aa48e&limit=10&recursive=true&includeItemTypes=Movie&includeItemTypes=Series&sortBy=IsFavoriteOrLiked&sortBy=Random&imageTypeLimit=0&enableTotalRecordCount=false&enableImages=false to 192.168.1.82 in 0:00:00.8187179 with Status Code 200
mediaserver-jellyfin-1  | [01:03:02] [INF] [64] TVHeadEnd.AccessTicketHandler: [TVHclient] AccessTicketHandler.GetAccessTicket: New ticket (#14) created for channelId=1243331310
mediaserver-jellyfin-1  | [01:03:02] [INF] [64] Jellyfin.Api.Helpers.MediaInfoHelper: User policy for sylvain. EnablePlaybackRemuxing: True EnableVideoPlaybackTranscoding: False EnableAudioPlaybackTranscoding: False
mediaserver-jellyfin-1  | [01:03:02] [INF] [64] Jellyfin.Api.Helpers.MediaInfoHelper: StreamBuilder.BuildVideoItem( Profile=Anonymous Profile, Path=http://192.168.1.20:9981/stream/channelid/1243331310?ticket=97258fa2ef54f82244be400d4335a5c137b9349e, AudioStreamIndex=null, SubtitleStreamIndex=null ) => ( PlayMethod=Transcode, TranscodeReason=ContainerNotSupported, VideoCodecNotSupported, AudioCodecNotSupported ) media:/videos/44b44066-02ed-4b8f-3972-23ae044b403f/master.m3u8?MediaSourceId=1243331310&VideoCodec=hevc,h264&AudioCodec=aac,mp3,ac3,eac3,flac,opus&AudioStreamIndex=-1&VideoBitrate=359360000&AudioBitrate=640000&api_key=<token>&TranscodingMaxAudioChannels=6&RequireAvc=false&SegmentContainer=mp4&MinSegments=2&BreakOnNonKeyFrames=True&TranscodeReasons=ContainerNotSupported,%20VideoCodecNotSupported,%20AudioCodecNotSupported
mediaserver-jellyfin-1  | [01:03:02] [INF] [97] Emby.Server.Implementations.Session.SessionManager: Playback stopped reported by app Swiftfin iOS 1.0.0 playing M6 bar le duc. Stopped at 0 ms

Xcode console:

[🔵 Debug] HomeViewModel#44:refresh() Refreshing home screen
didset orientations: UIInterfaceOrientationMask(rawValue: 2)
note: the section invalidation handler cannot currently mutate visible items for layouts containing estimated items. Please file an enhancement request on UICollectionView.
[🔵 Debug] LiveTVChannelsViewModel#178:startScheduleCheckTimer() LiveTVChannels schedule check...
didset orientations: UIInterfaceOrientationMask(rawValue: 26)
creating player instance using shared library
creating player instance using shared library
sent stop report

[sy6sy2]

Also, the "force direct play" boolean seems to be read only in one file (here https://github.com/jhays/Swiftfin/blob/2fe95a8de9ce4854b60669e05af989b45753db20/Shared/Extensions/JellyfinAPI/MediaSourceInfo%2BItemVideoPlayerViewModel.swift#L23) but, if I put a breakpoint here and if I start a live TV on my phone then this breakpoint is never met.

[jhays]

@sy6sy2 the force direct play flag you are referencing there isn't the one used by the Live TV feature- it has a separate force direct flag, .Experimental.liveTVForceDirectPlay, which I think should be getting checked in the liveVideoPlayerViewModel func used by Live TV.

I've added a bit of additional debug logging in my latest commit, as well as merged with the latest from main. If you can test again and share the Xcode logs, I'm hoping the latest logs might give me a clue regarding your issue.

[sy6sy2]

Thank you for your help. Unfortunately the latest changes seems to be broken, or it's me, the app is stuck on the splash screen on my device

[sy6sy2]

Ok .. rebooting the phone fixed the freeze issue -_-'

This is my log:

[🔵 Debug] HomeViewModel#44:refresh() Refreshing home screen
didset orientations: UIInterfaceOrientationMask(rawValue: 2)
note: the section invalidation handler cannot currently mutate visible items for layouts containing estimated items. Please file an enhancement request on UICollectionView.
didset orientations: UIInterfaceOrientationMask(rawValue: 26)
liveVideoPlayerViewModel response received
liveVideoPlayerViewModel resorting to first media source in the response
liveVideoPlayerViewModel matchingMediaSource being returned
creating player instance using shared library
creating player instance using shared library
[🔵 Debug] VideoPlayerManager#209:sendStopReport() sent stop report

[sy6sy2]

So, maybe I'm wrong and last time I checked I had my live TV using transcoding. Anyway, Swiftfin should be able to play my TVheadend channels without transcoding because the iOS VLC app is able to do it.

Currently, this is the log I get from my Jellyfin server when watching a channel from macOS:

mediaserver-jellyfin-1  | [18:12:05] [INF] [50] Jellyfin.Api.Helpers.MediaInfoHelper: StreamBuilder.BuildVideoItem( Profile=Jellyfin Media Player, Path=http://192.168.1.20:9981/stream/channelid/1243331310?ticket=6a94ebce2c98ec8cdd6656d11d06a4d9047d0bb6, AudioStreamIndex=null, SubtitleStreamIndex=null ) => ( PlayMethod=DirectPlay, TranscodeReason=0 ) media:/videos/44b44066-02ed-4b8f-3972-23ae044b403f/stream?MediaSourceId=1243331310&Static=true&AudioStreamIndex=-1&api_key=<token>

As we can see, no transcoding is using (also, the player media info confirms this).

With Swiftfin, this is what I get for the same channel:

ediaserver-jellyfin-1  | [18:13:59] [INF] [45] Jellyfin.Api.Helpers.MediaInfoHelper: StreamBuilder.BuildVideoItem( Profile=Anonymous Profile, Path=http://192.168.1.20:9981/stream/channelid/1243331310?ticket=17aea661e4cfecde31b3736285d3a8247b560ae2, AudioStreamIndex=null, SubtitleStreamIndex=null ) => ( PlayMethod=Transcode, TranscodeReason=ContainerNotSupported, VideoCodecNotSupported, AudioCodecNotSupported ) media:/videos/44b44066-02ed-4b8f-3972-23ae044b403f/master.m3u8?MediaSourceId=1243331310&VideoCodec=hevc,h264&AudioCodec=aac,mp3,ac3,eac3,flac,opus&AudioStreamIndex=-1&VideoBitrate=359360000&AudioBitrate=640000&api_key=<token>&TranscodingMaxAudioChannels=6&RequireAvc=false&SegmentContainer=mp4&MinSegments=2&BreakOnNonKeyFrames=True&TranscodeReasons=ContainerNotSupported,%20VideoCodecNotSupported,%20AudioCodecNotSupported

So, based on the device profile send by Swiftfin, Jellyfin server wants to transcode the channel. But I don't want that.

So I "hacked" DeviceProfileBuilder.swift with var directPlayProfiles: [DirectPlayProfile] = [DirectPlayProfile(type: .video)] to say something like "Hey server, no problem, I accept any video container or codec in direct play mode".

With this hack this is what I get from the server:

mediaserver-jellyfin-1  | [22:46:06] [INF] [53] Jellyfin.Api.Helpers.MediaInfoHelper: StreamBuilder.BuildVideoItem( Profile=Anonymous Profile, Path=http://192.168.1.20:9981/stream/channelid/1107113438?ticket=26020a577c9063161331de491b8ee4d94131ecb9, AudioStreamIndex=null, SubtitleStreamIndex=null ) => ( PlayMethod=DirectPlay, TranscodeReason=0 ) media:/videos/ee38274d-36df-73f9-057d-65bb01421819/stream?MediaSourceId=1107113438&Static=true&AudioStreamIndex=-1&api_key=<token>

Same line as with the macOS client. So I except the iOS player to be able to play this direct video flux but not, I only get a black screen 😕

Edit: Just for information, if I take the path returned by the server in the log and open it in iOS VLC app it just works. So I can confirm that Jellyfin server returns the correct path/URL.

[sy6sy2]

Ok I was able to play the live TV channel from TVHeadend on Swiftin without transcoding. It's really "hacky" but this is what I did --> https://github.com/sy6sy2/Swiftfin/commit/029af180781bfe63bc7b172437257cfd5e2bccfb

I guess we need to change some things to correctly handle this. Maybe @LePips knows something about that. I will try to have a look at https://github.com/jellyfin/jellyfin-media-player to check how live TV are handle

Edit: Ok, now it works, I am able to play any Live TV channel without transcoding on Swiftfin with the commit given above:

• Modification of directPlayProfiles in Shared/Objects/DeviceProfileBuilder.swift is used to force Jellyfin server to send the direct play URL. Without this modification we get a PlayMethod=Transcode, TranscodeReason=ContainerNotSupported, VideoCodecNotSupported, AudioCodecNotSupported from the server. I guess we need to update directPlayProfiles to add missing container/video codec. I will try to look at it to add the missing stuff.
• customPlaybackURL stuff is needed to send to VLCKit the correct media URL (the direct URL), without this modification, we give to VLCKit a Jellyfin URL. I guess we have something wrong here that need to be fixed.
• Finally, Shared/ViewModels/VideoPlayerViewModel.swift modification is needed to have audio working. Without this comment VLCKit player use a "ghost" audio track.

[jhays]

@sy6sy2 Thank you so much for debugging further to determine what was going on. Unfortunately I don't have an explanation for why Swiftfin is having issues with direct play with live TV. For now maybe we could use the approach on your branch, but maybe we could also add a flag to the buildProfile() func so that we can only apply these changes when it is called from the context of the liveVideoPlayerViewModel func. I'm not familiar enough with it to know if those changes could have any negative impact on other uses, so it might be safer to only have them apply for LiveTV when the live tv force direct flag is enabled.

[sy6sy2]

I'm pretty sure that my branch break some things, it's just a POC to demonstrate that Swiftfin is able to play TVheadend channels in direct play mode 😉 . Now that we know it works, I will take some time to correctly fix this issue without breaking anything else.

[sy6sy2]

Hey, this is my last test --> https://github.com/sy6sy2/Swiftfin/commit/927c50f7166c63f5466c90ae6ac7d67086707230

For me (at least with TVHeadend plugin) the "no transcoded" live TV mode works well with these changes. I tried to add my changes without breaking existing stuff.

Feel free to add my changes in your branch if you think it's ok for you. Also, if you need me to modify something feel free to ask. Thank you!

[sy6sy2]

Hi guys, any news on this? 😉

[jhays]

Hi guys, any news on this? 😉

Sorry for the delay, I've been tied up on other projects lately. But this is still on my mind as I use Swiftfin and the Live TV feature pretty much daily.

I've added your proposed fixes for direct play in the latest commit.

From my end, I think this PR is probably ready, unless there is any review feedback.

[LePips]

I've temporarily disabled livetv in #1013 to get an iOS update out fairly soon. I apologize that I haven't gotten to this PR but I've always been aware of it.

I've done a lot of work that would require a lot of this work to be refactored. For example,

• LiveTVChannelsViewModel can subclass PagingLibraryViewModel for better paging management
• logging system is by a service, so don't use NSLog
• Stateful which I will migrate over existing view models (that it is applicable for)
• new CollectionVGrid instead of CollectionView

I know I said somewhere that this would benefit from the video player refactor in that this could probably "have it's own overlay", but I didn't think that would require copying the entirety of the video players - I'm not going to manage that amount of just plain copied code.

Since I haven't looked at the video player stack for a while and will probably soon to migrate to Stateful along with a lot of stuff I've had on mind, I will see if the video player stuff could be cleaner as is. If not, I will investigate making it so.

Due to that, the next best steps for this work would probably be doing at the very least the four things listed, but I would probably also have additional comments in the future. Also, it may be best to split the view/view model fixes from the video player into 2 PRs.

I may also want to take a look at live tv since I want to clean it up, so I may get to the view stuff before you do - but I will open a draft PR if I do so don't feel pressured or like you can't do it.

You are welcome to continue in this PR or open a new one.