IDR Frame on each Segment Start?

[ProIcons]

System info

Operating System: Ubuntu 21.04 (K: 5.11.0-25-generic) Shaka Packager Version: packager-linux version v2.5.0-98b97709d7-release

I'm having the following workflow: I get an input from a Capture Card with ffmpeg, i'm reencoding it and i'm piping it to a shaka instance for dash packaging.

FFmpeg Command

ffmpeg -re -i <input> -threads 0 -movflags faststart -c:v libx264 -tune zerolatency -r 25 -g 160 -b:v 4300000 -force_key_frames expr:gte(t,n_forced*6.4) -min_seg_duration 6400000 -keyint_min 160 -pix_fmt yuv420p -s 1920x1080 -c:a aac -ar 48000 -ac 2 -ab 128000 -f mpegts udp://225.1.1.8:40929?reuse=1

Packager Command:

packager-linux 
  in=udp://225.1.1.8:40929?reuse=1,stream=video,init_segment=/home/nikolas/workspace/1/video_init.m4s,segment_template=/home/nikolas/workspace/1/video_$Number$.m4s 
  in=udp://225.1.1.8:40929?reuse=1,stream=audio,init_segment=/home/nikolas/workspace/1/audio_init.m4s,segment_template=/home/nikolas/workspace/1/audio_$Number$.m4s 
  --segment_duration 6.4 
  --min_buffer_time 12.8 
  --time_shift_buffer_depth 604800 
  --preserved_segments_outside_live_window 0 
  --mpd_output /home/nikolas/workspace/1/index.mpd

The segment size i chose for it was 6.4 seconds based on the blog i read about segment alignment on https://websites.fraunhofer.de/video-dev/why-and-how-to-align-media-segments-for-abr-streaming/ from Daniel Silhavy (developer of dash,js client)

So this will potentially run for months, one of our requirements is to be able to cut the manifests in arbitrary durations and converting them to static vods. So the way i have tackled this was to have a point of reference the actual Video Segments, (select for example 100 video segments) and properly offset the audio segments accordingly.

Thing is that apparently segments don't always start with an IDR Frame, and that causes some video frames to be missed when cutting.

This is an example snapshot of what linux-packager produces:

   4   │   <Period id="0" start="PT0S">
   5   │     <AdaptationSet id="0" contentType="audio" segmentAlignment="true">
   6   │       <Representation id="0" bandwidth="132447" codecs="mp4a.40.2" mimeType="audio/mp4" audioSamplingRate="48000">
   7   │         <AudioChannelConfiguration schemeIdUri="urn:mpeg:dash:23003:3:audio_channel_configuration:2011" value="2"/>
   8   │         <SegmentTemplate timescale="90000" initialization="audio_init.m4s" media="audio_$Number$.m4s" startNumber="1">
   9   │           <SegmentTimeline>
  10   │             <S t="126000" d="451200"/>
  11   │             <S t="577200" d="576000" r="47"/>
  12   │           </SegmentTimeline>
  13   │         </SegmentTemplate>
  14   │       </Representation>
  15   │     </AdaptationSet>
  16   │     <AdaptationSet id="1" contentType="video" width="1920" height="1080" frameRate="90000/3600" segmentAlignment="true" par="16:9">
  17   │       <Representation id="1" bandwidth="6428919" codecs="avc1.640028" mimeType="video/mp4" sar="1:1">
  18   │         <SegmentTemplate timescale="90000" initialization="video_init.m4s" media="video_$Number$.m4s" startNumber="1">
  19   │           <SegmentTimeline>
  20   │             <S t="126827" d="576000" r="44"/>
  21   │             <S t="26046827" d="489600"/>
  22   │             <S t="26536427" d="662400"/>
  23   │             <S t="27198827" d="576000" r="1"/>
  24   │           </SegmentTimeline>
  25   │         </SegmentTemplate>
  26   │       </Representation>
  27   │     </AdaptationSet>
  28   │   </Period>

Any ideas what i might do wrong? or ffmpeg/packager is doing wrong?

Edit: 9/8/21

If i use ffmpeg with these settings, and also instead of piping to shaka, use the dash muxer/demuxer it will work as intended

[Khepu]

I have spent countless hours trying to achieve a similar thing... I would greatly appreciate any info on this.

[cosmin]

It's certainly not expected for segments not to start with an IDR. There are some things in the ffmpeg command line that don't quite look right, like using -movflags faststart when the output is TS, using force key frames at 6.4 since it's already set with -g and -keyint_min, and -min_seg_duration 6400000 which seems out of place given the output type.

Can you reproduce this with latest ffmepg and latest shaka packager release?

[github-actions[bot]]

Closing due to inactivity. If this is still an issue for you or if you have further questions, the OP can ask shaka-bot to reopen it by including @shaka-bot reopen in a comment.