Implement stream resumption feature for handling disconnections

[zcesur]

Livestreams sometimes get interrupted due to network issues, OBS crash, server crash, streamer taking a break etc. When the user makes a new RTMP connection, our current pipeline initializes a new livestream on a new URL along with a new HLS playlist.

We should introduce a reconnection window (e.g. 1 hour) where we reconnect the user to their latest livestream.

To do that, we likely need to modify Membrane.HTTPAdaptiveStream.SinkBin to accept an existing playlist and initialize its state with that.

In addition we might need to add a discontinuity event/marker (EXT-X-DISCONTINUITY tag) at the end of the playlist so that the video player can play everything properly (as commonly used for mid-roll ads)

The player should seamlessly transition to the new segment without the user having to refresh the page.

[zcesur]

/bounty 400

[algora-pbc[bot]]

~~## 💎 $400 bounty • Algora~~

~~### Steps to solve:~~ ~~1. Start working: Comment /attempt #73 with your implementation plan~~ ~~2. Submit work: Create a pull request including /claim #73 in the PR body to claim the bounty~~ ~~3. Receive payment: 100% of the bounty is received 2-5 days post-reward. Make sure you are eligible for payouts~~

~~### ❗ Important guidelines:~~ ~~- To claim a bounty, you need to provide a short demo video of your changes in your pull request~~ ~~- If anything is unclear, ask for clarification before starting as this will help avoid potential rework~~ ~~- For assistance or questions, join our Discord~~

~~Thank you for contributing to algora-io/tv!~~

~~Add a bounty • Share on socials~~

Attempt	Started (GMT+0)	Solution
🟢 @Raushankumar-prog	Aug 29, 2024, 8:23:09 PM	WIP
🟢 @lastcanal	Oct 2, 2024, 6:13:17 PM	#113

[Raushankumar-prog]

/attempt #73

Options

• Cancel my attempt

[deepakdinesh1123]

@zcesur I setup algora on my local to work on this but when I try to livestream the stream gets cancelled as it cannot create a thumbnail and a new stream gets started, I think this is occurring due to running it on a mac M1 as rambo only supports x86_64 and requires rust compiler to use other architectures ( I have it installed but still running into error) just wanted to ask if you are using a mac arm based machine and how you have set it up

Edit:

I was able to set it up, let me know if you want me to add a section on how to setup in docs, could you please let me know what should be the value of the EVENT_SINK_URL

[zcesur]

Ooh interesting, yeah that'd be great to have in docs!

Re: EVENT_SINK_URL just leave it empty, filling out these env variables should be enough for almost all features

[deepakdinesh1123]

@zcesur I keep getting this error whenever I try to stream

[error] <0.860.0>/ Error occured in Membrane Pipeline:
%ArgumentError{message: "scheme is required for url: "}
    (finch 0.18.0) lib/finch/request.ex:135: Finch.Request.parse_url/1
    (finch 0.18.0) lib/finch/request.ex:103: Finch.Request.build/5
    (algora 0.1.0) lib/algora/library.ex:325: Algora.Library.toggle_streamer_live/2
    (algora 0.1.0) lib/algora/pipeline.ex:77: Algora.Pipeline.handle_child_notification/4
    (membrane_core 1.1.0) lib/membrane/core/callback_handler.ex:139: Membrane.Core.CallbackHandler.exec_callback/4
    (membrane_core 1.1.0) lib/membrane/core/callback_handler.ex:69: Membrane.Core.CallbackHandler.exec_and_handle_callback/5
    (membrane_core 1.1.0) lib/membrane/core/pipeline.ex:119: Membrane.Core.Pipeline.do_handle_info/2
    (membrane_core 1.1.0) lib/membrane/core/pipeline.ex:93: Membrane.Core.Pipeline.handle_info/2
    (stdlib 5.2.3) gen_server.erl:1095: :gen_server.try_handle_info/3
    (stdlib 5.2.3) gen_server.erl:1183: :gen_server.handle_msg/6
    (stdlib 5.2.3) proc_lib.erl:241: :proc_lib.init_p_do_apply/3

and in the stream I am seeing this error

[zcesur]

Just pushed a fix to main, can you try again? We were missing the sink_url != "" check

[deepakdinesh1123]

@zcesur it is working now thank you very much

[lastcanal]

This is possible to do without tearing down the pipeline; however, it will require upgrading membrane_rtmp_plugin to the latest version, which also requires Erlang 27.1 and Elixir 1.17. This version, released a week ago, completely changes the connection process and removes the Validator class, meaning the hackey GenServer calls back to the pipeline are no longer needed.

Using muxed_av for HLS is also not possible with reconnects because of the static pads on the muxer, so the pipeline will need to use separate_av in the HLS config and publish both audio and video tracks. This is currently incompatible with the single-manifest implementation of low-latency storage, although it could be extended to support multiple manifests, which also paves the way for supporting multiple bit-rates.

/attempt #73

Algora profile	Completed bounties	Tech	Active attempts	Options
@lastcanal	2 algora bounties	Elixir, JavaScript, C++ & more		Cancel attempt

[mat-hek]

Hey there! How is this going? Is there anything specific that Membrane team can help with?

[algora-pbc[bot]]

💡 @lastcanal submitted a pull request that claims the bounty. You can visit your bounty board to reward.

[lastcanal]

Hey @mat-hek!

I actually got reconnects working a few weeks ago but another project pulled my attention away before I could finish up. It was a fair amount of work but I ended up learning a lot about Membrane along the way! Building pipelines is a lot of fun once you figure out what's happening; I feel like I'm finally using the Erlang/OTP how it was meant to be used. Thanks!

I've upgraded to the latest version of membrane_rtmp_plugin, thankfully your recent changes to the connection architecture were exactly what was required for reconnects! Amazing! Also, the multiple-manifest low-latency storage adapter I added to Fishjam under the Apache license was ported to Algora and works great! Happy to send a PR to the Fishjam repo if you're interested.

I did run into static pad unlinking errors when reconnecting the RTMP Source, but managed to work around it by stacking an 'unlinkable' Tee in front of a modified Funnel that sends a notification on :end_of_stream instead of terminating. Is this a crazy way to do it? It seems to work great even for multiple reconnects.

Thanks for reaching out and offering a helping hand! I've pushed my WIP branch, any code review is appreciated! Lots of changes to Algora's pipeline :)

[mat-hek]

Hi @lastcanal, thanks for the PRs you did to Membrane! We'll have a look shortly. I'll try to have a look at the Algora one too.

To the point, an alternative approach to the problem would be to allow the entire pipeline to crash and then try to append to an existing manifest, for what we could use EXT-X-DISCONTINUITY, usually used for AD insertion. However, yours should also work and even have less impact on the stream (smaller data loss), at least unless the HLS or entire pipeline crashes.

Regarding the funnel, I think you can achieve this by setting the end_of_stream option. There's no notification though, but we could have that if it's needed. When you have a funnel, I guess it's enough for the HLS to be 'unlinkable' from dying RTMP and the tee is not needed 🤔

Regarding LL-HLS, I'm not familiar with that code, but I'd be happy to have it in the https://github.com/membraneframework/membrane_http_adaptive_stream_plugin if possible!

[lastcanal]

Thanks @mat-hek! The Membrane team is so incredibly responsive; it's been a pleasure contributing! Thanks to you guys i've been able to publish #116, a branch that offers multiple bit-rate manifests. Here is a link to the pipeline that does live-streaming better than Twitch, IMO, and all in 500 lines of code!

I tried the other end_of_stream options for the Funnel but only :never would work without taking down the whole pipeline. The only issue is I am unable to get an end-of-stream message from the rtmp sink, most likely because my rtmp sinks never actually terminate.. I will put together a PR for the funnel plugin explaining more why a :notify option would be useful, especially when working with static pads.

I will absolutely try to figure out how to add an LL-HLS storage option to the http adaptive stream plugin; that's a great idea! It would also be nice if the storage backends knew how to load manifests, insert a EXT-X-DISCONTINUITY and continue; that would save lots of compute resources, allowing us to shut down the pipeline after a much shorter 'reconnect' window. The current implementation I have works great for those network related disconnections, but is less efficient for longer duration disconnections.

Thanks again to you and the team for the amazing work on Membrane! It's been a blast learning to wrangle pipelines.

[mat-hek]

It would also be nice if the storage backends knew how to load manifests, insert a EXT-X-DISCONTINUITY and continue

Or pass the manifest to the sink, which would parse it, initialize its state and pass proper arguments to the storage backends 🤔

[algora-pbc[bot]]

🎉🎈 @lastcanal has been awarded $500! 🎈🎊

• Share on socials
• Give feedback