jim-nvenc RTMP video cannot be split correctly by ffmpeg HLS

[RinCat]

Operating System Info

Windows 11

Other OS

No response

OBS Studio Version

28.1.2

OBS Studio Version (Other)

No response

OBS Studio Log URL

https://obsproject.com/logs/SWbw_Yr__hpcnaDw

OBS Studio Crash Log URL

No response

Expected Behavior

jim-nvenc should generate a video stream that can be correctly segmented.

Current Behavior

The segmented HLS video by ffmpeg is not playable.

Steps to Reproduce

1. Use default OBS settings.
2. Set Encoder to NVIDIA NVENC H.264
3. Start a ffmepg with

ffmpeg -listen 1 -i rtmp://XXXXXXXX:1935/live/xxxxxxx -y -threads 2 -c:v copy -c:a copy -map "0:a?" -map "0:v?" -hls_time 4 -hls_list_size 15 -hls_flags delete_segments+independent_segments+program_date_time -hls_segment_filename /tmp/1111/%v-%06d.ts -master_pl_name master.m3u8 -f hls /tmp/1111/%v.m3u8

4. Start streaming.
5. Play the HLS files. ...

Anything else we should know?

I found this issue because one of self-hosted system use this way to do HLS and I found it not playable.

The issue exists in both 28.1.2 and 29 beta3, did not test others.

The issue not exists if fallback to ffmpeg in Windows, and not exists in Linux (because use system ffmpeg).

Only the first HLS is playable, all others get following error when use ffprobe and ffmpeg:

[h264 @ 0x64fabc3c4d00] non-existing SPS 0 referenced in buffering period
[h264 @ 0x64fabc3c4d00] non-existing PPS 0 referenced
[h264 @ 0x64fabc3c4d00] non-existing SPS 0 referenced in buffering period
[h264 @ 0x64fabc3c4d00] non-existing PPS 0 referenced
[h264 @ 0x64fabc3c4d00] decode_slice_header error
[h264 @ 0x64fabc3c4d00] no frame!
...
[mpegts @ 0x64fabc3a1980] decoding for stream 1 failed
[mpegts @ 0x64fabc3a1980] Could not find codec parameters for stream 1 (Video: h264 ([27][0][0][0] / 0x001B), none): unspecified size
Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options
Input #0, mpegts, from '0-000001.ts':
  Duration: 00:00:04.19, start: 5.571500, bitrate: 2609 kb/s
  Program 1 
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
  Stream #0:0[0x100]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 4 kb/s
  Stream #0:1[0x101]: Video: h264 ([27][0][0][0] / 0x001B), none, 60 fps, 60 tbr, 90k tbn, 180k tbc

Sample output: https://drive.google.com/drive/folders/1JtlwCEoSpVhmphS4z5ej_wBFn7kp98Rt?usp=sharing

[notr1ch]

I believe this is intended behavior, SPS/PPS do not need repeating in the RTMP specification as the type 0 packet containing AVC headers is intended to be stored and sent to clients by the RTMP server. If you set OBS to output HLS (or any other protocol) instead of RTMP, it will repeat SPS/PPS for each keyframe.

This could perhaps be considered as a bug in FFmpeg, as if it is acting as an RTMP server and transmuxing to HLS, it should probably be including the headers in each segment it writes.

[RinCat]

So the ffmpeg itself will repeating SPS/PPS when send to RTMP? If I let OBS fallback to built-in ffmpeg it wroks. This behavior is clearly inconsistent.

[notr1ch]

FFmpeg as an encoder will repeat SPS/PPS even when using RTMP where it is not supposed to be necessary. But FFmpeg as a muxer from e.g. RTMP to HLS will not use the SPS/PPS data from the RTMP header in the HLS segments, which seems like a bug.

[RinCat]

OK, I will file a bug to ffmpeg.

[jason-akw]

@RinCat Seems there is no issue on OBS 27.

[rAcHekLoS]

Same problem here since OBS 28 with NVENC encoder. If i set the colour format i420 instead of nv12 it works.

[RytoEX]

If I let OBS fallback to built-in ffmpeg it wroks.

@RinCat That is a different code path and different encoder implementation. Though, if I recall correctly, the code paths were consolidated somewhat in OBS Studio 28 and then further in 28.1.

Seems there is no issue on OBS 27.

@akw28888 See above. There was some code consolidation so that jim-nvenc and FFmpeg NVENC share more code.

What I would be interested in seeing are log files from OBS where it works and log files from OBS where it does not work so that we can verify what is occurring.

[RinCat]

I found this one still broken in 29.1 Beta 3.

[derrod]

I found this one still broken in 29.1 Beta 3.

You'll have to provide more details. This was an issue in FFmpeg and has been fixed upstream, so it ahould be fixed in our Windows, macOS, and Flatpak builds.

[RinCat]

It is OBS 29.1 Beta 3 I just downloaded from this github, Windows 11, NVENC H264.

17:04:49.968: [jim-nvenc: 'advanced_video_stream'] settings: 17:04:49.968: codec: H264 17:04:49.968: rate_control: VBR 17:04:49.968: bitrate: 10000 17:04:49.968: cqp: 20 17:04:49.968: keyint: 250 17:04:49.968: preset: p3 17:04:49.968: tuning: hq 17:04:49.968: multipass: qres 17:04:49.968: profile: high 17:04:49.968: width: 2560 17:04:49.968: height: 1440 17:04:49.968: b-frames: 2 17:04:49.968: lookahead: false 17:04:49.968: psycho_aq: true 17:04:49.968: 17:04:50.019: ---------------------------------

Play the ts file:

ffprobe 0-000001.ts
ffprobe version 6.0 Copyright (c) 2007-2023 the FFmpeg developers
  built with clang version 15.0.7
  configuration: --prefix=/usr --libdir=/usr/lib64 --shlibdir=/usr/lib64 --docdir=/usr/share/doc/ffmpeg-6.0-r1/html --mandir=/usr/share/man --enable-shared --cc=clang --cxx=clang++ --ar=llvm-ar --nm=llvm-nm --strip=x86_64-pc-linux-gnu-strip --ranlib=llvm-ranlib --pkg-config=x86_64-pc-linux-gnu-pkg-config --optflags='-march=znver3 -O2 -pipe' --disable-static --enable-avfilter --disable-stripping --disable-optimizations --disable-libcelt --disable-indev=v4l2 --disable-outdev=v4l2 --disable-indev=oss --disable-indev=jack --disable-indev=sndio --disable-outdev=oss --disable-outdev=sndio --enable-version3 --enable-nonfree --enable-bzlib --enable-runtime-cpudetect --disable-debug --disable-gcrypt --enable-gnutls --enable-gmp --enable-gpl --disable-hardcoded-tables --enable-iconv --disable-libxml2 --enable-lzma --enable-network --disable-opencl --disable-openssl --enable-postproc --disable-libsmbclient --enable-ffplay --enable-sdl2 --enable-vaapi --enable-vdpau --enable-vulkan --enable-xlib --enable-libxcb --enable-libxcb-shm --enable-libxcb-xfixes --enable-zlib --disable-libcdio --disable-libiec61883 --disable-libdc1394 --disable-libcaca --disable-openal --enable-opengl --disable-libv4l2 --enable-libpulse --enable-libdrm --disable-libjack --disable-libopencore-amrwb --disable-libopencore-amrnb --disable-libcodec2 --enable-libdav1d --enable-libfdk-aac --enable-libopenjpeg --enable-libjxl --enable-libbluray --disable-libgme --disable-libgsm --disable-libaribb24 --disable-mmal --disable-libmodplug --enable-libopus --enable-libvpl --disable-libilbc --disable-librtmp --disable-libssh --disable-libspeex --disable-libsrt --enable-librsvg --enable-ffnvcodec --enable-libvorbis --enable-libvpx --disable-libzvbi --disable-appkit --disable-libbs2b --disable-chromaprint --disable-cuda-llvm --disable-libflite --disable-frei0r --disable-libvmaf --disable-libfribidi --enable-fontconfig --disable-ladspa --enable-lcms2 --enable-libass --disable-libplacebo --disable-libtesseract --disable-lv2 --enable-libfreetype --disable-libvidstab --disable-librubberband --disable-libzmq --enable-libzimg --enable-libsoxr --enable-pthreads --disable-amf --disable-libvo-amrwbenc --disable-libkvazaar --disable-libaom --enable-libmp3lame --disable-libopenh264 --disable-librav1e --disable-libsnappy --enable-libsvtav1 --enable-libtheora --disable-libtwolame --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --disable-armv5te --disable-armv6 --disable-armv6t2 --disable-neon --disable-vfp --disable-vfpv3 --disable-armv8 --disable-mipsdsp --disable-mipsdspr2 --disable-mipsfpu --disable-altivec --disable-vsx --disable-power8 --disable-amd3dnow --disable-amd3dnowext --disable-fma4 --disable-xop --cpu=znver3 --disable-doc --disable-htmlpages --enable-manpages
  libavutil      58.  2.100 / 58.  2.100
  libavcodec     60.  3.100 / 60.  3.100
  libavformat    60.  3.100 / 60.  3.100
  libavdevice    60.  1.100 / 60.  1.100
  libavfilter     9.  3.100 /  9.  3.100
  libswscale      7.  1.100 /  7.  1.100
  libswresample   4. 10.100 /  4. 10.100
  libpostproc    57.  1.100 / 57.  1.100
[h264 @ 0x555ebcec2900] non-existing SPS 0 referenced in buffering period
[h264 @ 0x555ebcec2900] non-existing PPS 0 referenced
[h264 @ 0x555ebcec2900] non-existing SPS 0 referenced in buffering period
[h264 @ 0x555ebcec2900] non-existing PPS 0 referenced
[h264 @ 0x555ebcec2900] decode_slice_header error
[h264 @ 0x555ebcec2900] no frame!
[h264 @ 0x555ebcec2900] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0x555ebcec2900] decode_slice_header error
[h264 @ 0x555ebcec2900] no frame!
[h264 @ 0x555ebcec2900] non-existing PPS 0 referenced
    Last message repeated 1 times

[Fenrirthviti]

Please provide the entire log file from the test.

[RinCat]

Log: https://obsproject.com/tools/analyzer?log_url=https%3A%2F%2Fobsproject.com%2Flogs%2FtvnB2oOs9T9U4Us0#logURL

Verbose: https://obsproject.com/tools/analyzer?log_url=https%3A%2F%2Fobsproject.com%2Flogs%2FIMwlL1tsSRyMQJBY#logURL

Video sample: tmp.zip

[RinCat]

I still think this maybe a bug in jim-nvenc. If I change the color format from NV12 to I420 or I444, it will works.

[rAcHekLoS]

Same problem here with NV12, the first chunk for HLS is playable and all other chunks are unplayalbe:

Log: https://obsproject.com/logs/6nR36nbdhTd860S2

Video sample: hls_test.zip

[notr1ch]

Your log shows you using RTMP, not HLS. As already stated, it is not a requirement to repeat SPS/PPS for RTMP, as the server is responsible for storing this from the initial connection and providing it to clients. If you transmux RTMP to HLS or any other format with individual segments, then the transmux process should be responsible for ensuring that each segment gets a copy of the SPS/PPS from the RTMP stream.

[rAcHekLoS]

Nevertheless, the problem is only present since OBS 28.0.0 and only occurs with NVENC in combination with the colour format NV12. With OBS < 28.0.0 it works without problems.