Dash representation

[ariselseng]

Hi, Do you have any interest for this? https://github.com/joshmarshall/nginx-rtmp-module/pull/1

[sergey-dryabzhinsky]

Not quite understand the whole idea with 'representation'. Can you provide some explanation?

[ariselseng]

Its the dash equivalent to hls_variant, giving us adaptive streaming.

[sergey-dryabzhinsky]

If it's like http://nginx-rtmp.blogspot.ru/2013/07/hls-variant-playlist.html and working then it's very interesting.

[AlexRevin]

Agree this could be merged here

[sergey-dryabzhinsky]

Suggest to convert option dash_representation to dash_variant to make it correspond to hls options.

[Bond246]

Hello, is the function already integrated?

[sergey-dryabzhinsky]

Not yet.

[joshmarshall]

Hi folks! ( @sergey-dryabzhinsky and @cowai )

GitHub is not sending me email notifications unfortunately so I missed these notes. There are some optimizations needed to make this more efficient at building out the DASH representations, and I'd need to merge in recent changes (this PR is a bit old). I'd be happy to change "dash_representation" to "dash_variant" if that's more intuitive to folks as well.

I know @stephenbasile had some thoughts on the optimization front, I'll need to re-connect with him to make sure those are captured.

[sergey-dryabzhinsky]

@joshmarshall Thank you for response!

[joelio]

+1 for this! Any movement so far? Thanks

[Prendo93]

This would be awesome, +1.

[DJaeger]

:+1: @joshmarshall: Would be great, if you can fix nested DASH playlist before

[TimIsOverpowered]

+1

[MoffTigriss]

I worked on the adaptation of https://github.com/joshmarshall/nginx-rtmp-module/pull/1 for the (almost) current dev branch. I need that capability for my project.

My curent work is here (it's messy, not compilable, but probably not complicated to correct. I need some free time to do it myself) : https://github.com/MoffTigriss/nginx-rtmp-module/tree/work-dash-5

If someone want to redo the full work, it's not complicated : the most complex task is converting the dash template to a most grainy one. And using the dash_template.h (wich is a really good idea).

[ReubenM]

I think there is an big limiting factor with all the "DASH representation", "HLS Variant", and "RTMP dynamic streaming". (I think you could add this as a limitation to "MP4-SVC" as well.) The problem is (assuming I understand documentation) is that to do it correctly, the different spatial versions of the same encoding all need to have the same I-P-B frame sequence and for the segmented formats that they are all segmented the same. (Maybe the P and B frame types do not need to match, but the I-frames do) The only way I know of doing this is to lock down the frame type decisions to a static sequence, with a static segmentation size. This really sucks because it disables a lot of optimizations for both size and quality in allowing the encoder to decide frame type according to content (for instance, inserting key frames on scene changes). But without a locked down frametype sequence, there is no means of being sure that the different spatial variants produced by the encoder will all have keyframes and segmentation in the same places.

I assume most people using this are encoding using x264, and what is needed is for x264 to have a way to provide a master-slave IPC between different encoding processes so that the frame-type decisions of the master encoder processes can be copied by slave encoder processes.

[matiaspl]

I was worried about the same issues ten years ago when I was using multibitrate windows media streaming. But since then bandwidth got dirt cheap.

Sure, using fixed GOP size is suboptimal in terms of bitrate allocation (and for Apple compliant HLS fixed frame count per segment is to be expected). Suboptimal doesn't mean it sucks badly, and it's easy to force I frame placement without even disabling scene detection algos. So for example you can have multiple I frames for 20-30 second long segments. This way however, you force the client to cache a lot of supposedly unneeded data and the streaming starts usually after 3 first segments are fully downloaded, which might hurt mobile users' data plan more than shorter and less optimaly encoded segments would.

Everyone does multi bitrate streaming these days and it just works. That's why we too should have Dash variants working :)

Matt

12 gru 2016 18:43 "Reuben Martin" [email protected] napisał(a):

I think there is an big limiting factor with all the "DASH representation", "HLS Variant", and "RTMP dynamic streaming". (I think you could add this as a limitation to "MP4-SVC" as well.) The problem is (assuming I understand documentation) is that to do it correctly, the different spatial versions of the same encoding all need to have the same I-P-B frame sequence and for the segmented formats that they are all segmented the same. (Maybe the P and B frame types do not need to match, but the I-frames do) The only way I know of doing this is to lock down the frame type decisions to a static sequence, with a static segmentation size. This really sucks because it disables a lot of optimizations for both size and quality in allowing the encoder to decide frame type according to content (for instance, inserting key frames on scene changes). But without a locked down frametype sequence, there is no means of being sure that the different spatial variants produced by the encoder will all have keyframes and segmentation in the same places.

I assume most people using this are encoding using x264, and what is needed is for x264 to have a way to provide a master-slave IPC between different encoding processes so that the frame-type decisions of the master encoder processes can be copied by slave encoder processes.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/sergey-dryabzhinsky/nginx-rtmp-module/issues/55#issuecomment-266498813, or mute the thread https://github.com/notifications/unsubscribe-auth/AB049tjj9BOkwxlrux2hyd8e-ADTbCdFks5rHYfJgaJpZM4GsGeZ .

[ut0mt8]

Funny I have the same need to have some variant in my dash live stream. I quickly hack something working this afternoon (a very static one). I will review your work tomorrow MoffTigriss to make something more robust and configurable.

[MoffTigriss]

ut0mt8 : nice ! If you want a clean base, look that branch instead : https://github.com/MoffTigriss/nginx-rtmp-module/tree/work-dash-4 I f*cked up the manual merging with the lasts series of patchs from Sergey in work-dash-5 (the logic for the times variables changed, and the template too, and I hadn't enough time to study the implications).

[joshmarshall]

Hi folks --

Sorry for the long silence on this. @MoffTigriss I'm happy to help (extra pair of eyes, etc) since it looks like you are taking ownership of the feature! I can close my PR and you can open one so we can start commenting on that, if you'd like?

[ut0mt8]

Well I will try to post my solution tomorrow, but my implementation was far more simple. The idea is just to write a custom mpd in addition to some regular one. Example:

 application livestream {
            live on;
            dash on;
            dash_path /dev/shm/dash;
            dash_nested off;

            dash_variant _low BANDWIDTH=160000;
            dash_variant _mid BANDWIDTH=320000;
            dash_variant _hi  BANDWIDTH=640000;
        }

So we can receive 3 steam ; livestream_low, livestream_mid, livestream_hi write the 3 manifests normally, and write an extra manifest livestream.mpd which contains all the variant.

This is a bit tricky but should work. The only point is to correctly match the context name to avoid writing three time the extra mpd.

[ut0mt8]

btw nested dash will need extra effort but who care ?

[MoffTigriss]

@joshmarshall That can be a good thing ! I need to finish the adaptation to the current dev branch status before (or if someone want to do it), but yeah, an updated PR is a good idea.

@ut0mt8 The concatenation of different manifests descriptions from simultaneous encodings is enough to have a multi-bitrate manifest ?

[ut0mt8]

Not exactly : but something like this is sufficient :

...
<AdaptationSet
        mimeType="video/mp4"
        startWithSAP="1"
        segmentAlignment="true">
      <Representation
          id="live_2_video_1"
          mimeType="video/mp4"
          codecs="avc1.42c01e"
          width="640"
          height="360"
          bandwidth="500000">
        <SegmentTemplate
            presentationTimeOffset="0"
            timescale="1000"
            media="live_1-$Time$.m4v"
            initialization="live_1-init.m4v">
          <SegmentTimeline>
             <S t="7713322" d="6120"/>
          </SegmentTimeline>
        </SegmentTemplate>
      </Representation>
      <Representation
          id="live_2_video_2"
          mimeType="video/mp4"
          codecs="avc1.42c01e"
          width="640"
          height="360"
          bandwidth="250000">
        <SegmentTemplate
            presentationTimeOffset="0"
            timescale="1000"
            media="live_2-$Time$.m4v"
            initialization="live_2-init.m4v">
          <SegmentTimeline>
             <S t="7713322" d="6120"/>
          </SegmentTimeline>
        </SegmentTemplate>
      </Representation>
    </AdaptationSet>

This can be easily generated from one or another real stream (assuming we insert static bandwith and w/h)

[ut0mt8]

Work in progress at home. I think I will have something clean to show next week.

[ut0mt8]

So you can check my version at https://github.com/ut0mt8/nginx-rtmp-module/ which it basically working.

A live demo flux is available here "http://54.93.218.190/dash/live.mpd". It work/test with shakaplayer and dashjs which was quite sufficient for me.

Implementation note : this is principally copy/pasta from the hls variant code. It work the same way using the nginx configuration file to construct the variant manifest.

Some things can be easily fixed : the dash nested support is broken for now but it was trivial to add it. side question : do we need having multiple audio representation (need more variable , parsing ..)

Note : a better way to construct the variant manifest is to store/shared configuration from the different flux (using file like in the initial attempt is an idea , but I don't really like it), is there another way ?

[ut0mt8]

Ahh and any review / pair of eyes are welcome.

[ut0mt8]

btw I fix in a way the dash_nested option on my variant fork. It was clean, but does not separate the variant mpd wich was on the root of the path. seems clean to me.

[hunternet93]

@ut0mt8 Your fork is working perfectly for me so far! I haven't fully tested everything yet, but it's working well with the dash.js v2.4.1 reference player. I'm using a 500 kbps 640x360 _low, 1500 kbps 1280x720 _med, and 5000 kbps 1920x1080 _high streams.

Adaptive streaming is a required feature for me since many of my viewers have awful internet connections (it's the way things are in rural USA), so being able to auto-switch bitrates is awesome.

[ut0mt8]

Thanks. If I have some time I had to find a better way to create the multibirate manifest. Currently I dig into to the code to handle Ad insertion from rtmp metadata to dash; and still working on implementing Common encryption (but this not as easy as I figured at the beginning)

[Bond246]

Hello,

for me audio embedding from sub-mpd to master.mpd seems not to work. It is inside my master_hi.mpd but not in master.mpd.

First moments after start it works in sub-mpd and merged mpd but after some secounds the audio representation gets lost in merged mpd. Any ideas?

application ingest {
                live on;
                exec ffmpeg -i rtmp://127.0.0.1/$app/$name -map 0:0 -map 0:1 -s 1280x720 -c:v libx264 -b:v 3284k -preset medium -x264opts keyint=75:scenecut=-1 -c:a aac -b:a 192k -bufsize 3800k -f flv rtmp://127.0.0.1/dash/$name_hi -map 0:0 -s 960x540 -c:v libx264 -b:v 1800k -preset medium -x264opts keyint=75:scenecut=-1 -bufsize 3000k -f flv rtmp://127.0.0.1/dash/$name_med -map 0:0 -s 640x360 -c:v libx264 -b:v 1024k -preset medium -x264opts keyint=75:scenecut=-1 -bufsize 1500k -f flv rtmp://127.0.0.1/dash/$name_low;
        }

application dash {
                live on;
                dash on;
                dash_nested off;
                dash_path /home/fhe/nginx-webroot/mpeg_dash;
                dash_fragment 6;
                dash_playlist_length 120;
                dash_cleanup on;
                dash_variant _hi bandwidth="3584000" width="1280" height="720";
                dash_variant _med bandwidth="1800000" width="960" height="540";
                dash_variant _low bandwidth="1240000" width="640" height="360";
        }