Add Twitch/Streamlink as a media source

[Starz0r]

In a similar vein to https://github.com/chrisbenincasa/tunarr/issues/179, being able to add Twitch, or any compatible source via Streamlink, would be great as well. Some media systems like Jellyfin allow you to pull from Streamlink compatible sources via a plugin, so you'd also be able to access it from there in some cases as well. Since Streamlink exists, it's also likely a lot easier to implement since it could (probably) easily be piped into FFmpeg. Allowing any compatible source from Streamlink does present some immediate problems.

1. Length isn't always known ahead of time.

Unlike other broadcast sources, you may not always know when the stream will end. If you're using the web guide, then we'll need to display information to the user accurately about what we do know. We could just display that the stream is "infinite" until it ends, so it'll show up as going off into the distance or that it's length is "indeterminate". Or, we could show that it's "Live", but only extend it out in time-blocked chunks (15-minutes, 30-minutes, etc). This should probably be configurable.

Also, even for schedules we do know, we have no clue if they're accurate or if they'll adhere to them, so there needs to be safeguards to ensure that if a broadcast does go for longer (or shorter) that future scheduling isn't messed up.

2. Scheduling interleaved content would be a slight pain

Imagine this: A regular media source is playing on the channel when a broadcast goes live. If that media source hasn't finished, what should realistically happen here? Obviously, this should be configurable, but I can think of many more situations that are more ambiguous and need more tuning. Like, you have two streams authorized to play on a single channel, but which should get chosen? First to cast? The one with higher priority? There is probably way more situations than just this, and they all likely have to be handled in some way.

[chrisbenincasa]

This is an interesting idea to explore (and has been brought up before in reference to Youtube via yt-dlp). As you mentioned there are definitely some intricacies to work out regarding scheduling and I'm sure that there would be nuances to getting the streaming working as well.

I suppose one question I have before diving deeper in how this would work would be - is there a reason that an m3u aggregator like Threadfin wouldn't work for this case? Note it's certainly possible it wouldn't work with Streamlink, I'm just wondering... My initial hesitation comes from this type of feature teetering on the edge of making Tunarr a "stream proxy", which IMO is currently out of the scope of the project. I think I'd like to know some details on why Tunarr is the right home for something like that.

[Starz0r]

I think I'd like to know some details on why Tunarr is the right home for something like that.

I believe it's the right reason for the same reason that having yt-dlp support would be. Having it built-in usually means a better and closer integration to core systems. It also saves on having to run additional external services.

I suppose one question I have before diving deeper in how this would work would be - is there a reason that an m3u aggregator like Threadfin wouldn't work for this case?

Haven't heard of Threadfin until now. Upon a cursory surface level glance, it does not appear like this would work? It appears like you'd somehow have to get Streamlink to expose a m3u URI or pipe the Streamlink data into FFmpeg, which can then convert it into a m3u compatible format, then use a web proxy to expose that URI. Which sounds like a lot of finagling just to use Threadfin.

Is there some other reason why you believe a m3u aggregator might be a good alternative to having this is a native Tunarr feature?

[chrisbenincasa]

I believe it's the right reason for the same reason that having yt-dlp support would be. Having it built-in usually means a better and closer integration to core systems. It also saves on having to run additional external services.

This is fair, though I wasn't totally suggesting that YT was a solid idea either. That said, it at least has the benefit of discrete programming, which makes everything a whole lot easier to reason about (assuming we ignore YT live streams).

Haven't heard of Threadfin until now. Upon a cursory surface level glance, it does not appear like this would work? It appears like you'd somehow have to get Streamlink to expose a m3u URI or pipe the Streamlink data into FFmpeg, which can then convert it into a m3u compatible format, then use a web proxy to expose that URI. Which sounds like a lot of finagling just to use Threadfin.

I didn't look deeply into it; this more of an off-hand comment that it seemed like something that was more apt this type of ask. It does seem like Streamlink would have to support some sort of HLS style output, which it apparently doesn't (best I can see from a quick look at the docs is output to a single file). It does seem to have the ability to just reproduce the underlying stream's HLS playlist, which would get you close, but I didn't dig much further than that.

Is there some other reason why you believe a m3u aggregator might be a good alternative to having this is a native Tunarr feature?

The reason I suggested was that to me the underlying ask is "expose these streams in my EPG"; using a proxy + aggregator seemed like a more straightforward way to do this. I'm wary of Tunarr simply becoming a stream proxy and veering too far off the path of "recreating TV". OTOH I do so some cool opportunities with expanding the different types of content you can pull into Tunarr.

---

All that said, I think it's worthwhile to explore the feature, and the general idea of allowing live stream sources for programming, since it's unique and has been brought up several times (so there is clearly some desire). Going back to your original post and how to tackle some of nuances of indeterminate programming: we do have something similar in place today, which is on-demand channels (playback timer is paused when not viewing). This shows up on the guide as a single block with a "paused" indicator. The schedule is calculated when you start the stream and then is frozen again when the stream ends. I could see this meshing well into this feature as well.

I think the thing that seems most abstract to me right now is how we'd fit these into our transcode pipeline. Currently this requires effectively ffprobe'ing the input to gather metadata about the program stream in order to decide how to transcode. It's unclear to me how we'd get this data in this case; we may have to implement something that briefly buffers the stream onto disk in order to probe it.

[Starz0r]

I'm wary of Tunarr simply becoming a stream proxy and veering too far off the path of "recreating TV"

Oh no, I totally understand not wanting to give in to the feature bloat. However, having native yt-dlp support allows you to save storage space by only needing to re-encode. In a similar way Streamlink allows you to do the same. Of course I could probably jerry-rig a setup to download a live broadcast to a file and then expose that in Jellyfin, but the integration would be way way looser than if it was natively implemented.

I think the thing that seems most abstract to me right now is how we'd fit these into our transcode pipeline. Currently this requires effectively ffprobe'ing the input to gather metadata about the program stream in order to decide how to transcode. It's unclear to me how we'd get this data in this case; we may have to implement something that briefly buffers the stream onto disk in order to probe it.

There shouldn't be any need to place buffers on disk, even briefly. I believe Streamlink has a way to get metadata about broadcasts by just providing it a valid URI. It would usually be more of: "this broadcast does not provide any metadata to the program", in which case you might have to fallback to ffprobe. Most URIs though should provide the level of metadata you need without ffprobing.

[chrisbenincasa]

Sorry for the delay here, but I've made the decision to look into supporting arbitrary remote streams / inputs. So this should be coming at some point

[Starz0r]

No worries!

I saw that the size:xl tag got added, and I figured it might be a bit before I even see the fruits of this. Which is fair, since it does seem to be a rather large ask, but I'm glad you're deciding to tackle it!

Something I would like to leave here, is some details of what I remember from digital TV Guides when a stream was involved:

1. Livestreams would sometimes, but not always, expand into infinity. Typically you'd see this if the channel did not know exactly how long the event would go on for.
2. Estimated or provided guesses meant you could see when they would "end", but I remember seeing quite a few events that would spill over when they were suppose to end. How you handle this is up to you, but what I saw is it would say that whatever was scheduled for that time slot would be playing, but in reality, it was the live event still.