AlexxIT
Challenges using ffmpeg to copy & stream an HLS/fMP4 stream to RTP (but works with an HLS/TS stream) -- Octoprint/Obico questions
Octoprint (for 3D printing) seems to best support HLS for mp4 (at least out of the box, WebRTS and MSE don't seem to work). I am using a stream like the following for my PiCamera v3 on a RPi5
picam_h264:
- exec:libcamera-vid -t 0 --libav-format h264 -o -
The streaming seems to have less latency when using HLS/fMP4 (stream_url = http://mypi:1984/api/stream.m3u8?src=picam_h264&mp4)
rather than HLS/TS ( stream_url = http://mypi:1984/api/stream.m3u8?src=picam_h264).
However for the obico plugin, I need to use ffmpeg to copy the stream to an rtp Janus stream using something like:
ffmpeg -re -i <stream url> -an -c:v copy -an -f rtp rtp://127.0.0.1:17732?pkt_size=1300
However, ffmpeg terminates when using HLS/fMP4 after about 3 frames (though works fine with the lesser quality HLS/TS stream)
ffprobe shows the following differences between the streams: HLS/TS:
Input #0, hls, from 'http://mypi:1984/api/stream.m3u8?src=picam_h264':
Duration: N/A, start: 0.000000, bitrate: N/A
Program 0
Metadata:
variant_bitrate : 192000
Stream #0:0: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(tv, bt709), 1536x864, 15 tbr, 90k tbn
Metadata:
variant_bitrate : 192000
HLS/fMP4:
Input #0, hls, from 'http://mypi:1984/api/stream.m3u8?src=picam_h264&mp4':
Duration: N/A, start: 0.000000, bitrate: 1248005 kb/s
Program 0
Metadata:
variant_bitrate : 192000
Stream #0:0: Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709), 1536x864, 1248005 kb/s, 90k tbr, 90k tbn (default)
Metadata:
variant_bitrate : 192000
major_brand : iso5
minor_version : 512
compatible_brands: iso5iso6mp41
- Why does ffmpeg fail to copy and stream to RTP with HLS/fMP4 but succeeds with HLS/TS?
- Are there any parameters/filters I can add to ffmpeg (short of re-transcoding) that will allow it to work with HLS/fMP4?
Thanks
How do I find that out? No error message is printed on the console...
This is the output
# ffmpeg -re -i "http://mypi:1984/api/stream.m3u8?src=picam_h264&mp4" -c:v copy -an -f rtp rtp://127.0.0.1:17732?pkt_size=1300
ffmpeg version 5.1.4-0+rpt3+deb12u1 Copyright (c) 2000-2023 the FFmpeg developers
built with gcc 12 (Debian 12.2.0-14)
configuration: --prefix=/usr --extra-version=0+rpt3+deb12u1 --toolchain=hardened --incdir=/usr/include/aarch64-linux-gnu --enable-gpl --disable-stripping --disable-mmal --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libglslang --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librist --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sand --enable-sdl2 --disable-sndio --enable-libjxl --enable-neon --enable-v4l2-request --enable-libudev --enable-epoxy --libdir=/usr/lib/aarch64-linux-gnu --arch=arm64 --enable-pocketsphinx --enable-librsvg --enable-libdc1394 --enable-libdrm --enable-vout-drm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-libplacebo --enable-librav1e --enable-shared
libavutil 57. 28.100 / 57. 28.100
libavcodec 59. 37.100 / 59. 37.100
libavformat 59. 27.100 / 59. 27.100
libavdevice 59. 7.100 / 59. 7.100
libavfilter 8. 44.100 / 8. 44.100
libswscale 6. 7.100 / 6. 7.100
libswresample 4. 7.100 / 4. 7.100
libpostproc 56. 6.100 / 56. 6.100
[hls @ 0x5555861cd230] Opening 'http://mypi:1984/api/hls/playlist.m3u8?id=zYde7mSn' for reading
[hls @ 0x5555861cd230] Skip ('#EXT-X-VERSION:6')
[hls @ 0x5555861cd230] Opening 'http://mypi:1984/api/hls/init.mp4?id=zYde7mSn' for reading
[hls @ 0x5555861cd230] Opening 'http://mypi:1984/api/hls/segment.m4s?id=zYde7mSn&n=0' for reading
[hls @ 0x5555861cd230] Opening 'http://mypi:1984/api/hls/segment.m4s?id=zYde7mSn&n=1' for reading
[http @ 0x5555861d4250] Opening 'http://mypi:1984/api/hls/playlist.m3u8?id=zYde7mSn' for reading
[hls @ 0x5555861cd230] Skip ('#EXT-X-VERSION:6')
[hls @ 0x5555861cd230] skipping 1 segments ahead, expired from playlists
[http @ 0x555586202230] Opening 'http://mypi:1984/api/hls/init.mp4?id=zYde7mSn' for reading
[hls @ 0x5555861cd230] Opening 'http://mypi:1984/api/hls/segment.m4s?id=zYde7mSn&n=3' for reading
Input #0, hls, from 'http://mypi:1984/api/stream.m3u8?src=picam_h264&mp4':
Duration: N/A, start: 0.000000, bitrate: 1299552 kb/s
Program 0
Metadata:
variant_bitrate : 192000
Stream #0:0: Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709), 1536x864, 1299552 kb/s, 90k tbr, 90k tbn (default)
Metadata:
variant_bitrate : 192000
major_brand : iso5
minor_version : 512
compatible_brands: iso5iso6mp41
Output #0, rtp, to 'rtp://127.0.0.1:17732?pkt_size=1300':
Metadata:
encoder : Lavf59.27.100
Stream #0:0: Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709), 1536x864, q=2-31, 1299552 kb/s, 90k tbr, 90k tbn (default)
Metadata:
variant_bitrate : 192000
major_brand : iso5
minor_version : 512
compatible_brands: iso5iso6mp41
SDP:
v=0
o=- 0 0 IN IP4 127.0.0.1
s=No Name
c=IN IP4 127.0.0.1
t=0 0
a=tool:libavformat LIBAVFORMAT_VERSION
m=video 17732 RTP/AVP 96
b=AS:1299552
a=rtpmap:96 H264/90000
a=fmtp:96 packetization-mode=1; sprop-parameter-sets=Z2QAKKzsBgBtpqAgICgAAAMACAD0JAB4wYnA,aO4Pyw==; profile-level-id=640028
Stream mapping:
Stream #0:0 -> #0:0 (copy)
Press [q] to stop, [?] for help
frame= 1 fps=0.0 q=-1.0 Lsize= 162kB time=00:00:00.00 bitrate=120780363.6kbits/s speed=0.00308x
video:160kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 1.110509%
#
So it just seems to terminate without error
Could it be there is something wrong with the HLS/fMP4 module? I just tried to h264 transcode the output using:
ffmpeg -re -i "http://mypi:1984/api/stream.m3u8?src=picam_h264&mp4" -c:v libx264 -an output.h264
or
ffmpeg -re -i "http://mypi:1984/api/stream.m3u8?src=picam_h264&mp4=flac" -c:v libx264 -an output.h264
which terminates, while the HLS/TS version transcodes fine:
ffmpeg -re -i "http://mypi:1984/api/stream.m3u8?src=picam_h264" -c:v libx264 -an output.h264
Similarly even a simple download to a file works for HLS/TLS
ffmpeg -re -i "http://mypi:1984/api/stream.m3u8?src=picam_h264" output.m3u8
but fails for HLS/MP4:
ffmpeg -re -i "http://mypi:1984/api/stream.m3u8?src=picam_h264&mp4" output.m3u8
Interestingly though ffplay works on both HLS/TLS and HLS/fMP4
I verified all the above on 2 different cameras (one a PiCam v3 and another a USB cam)
So wondering if perhaps the HLS/fMP4 format is "non-standard" in the sense that ffmpeg is not able to properly parse it whether to transcode, copy or even just download the stream.
OK - this is weird - I tried on several other Linux systems and they all worked there. It only seems to fail on my RPi5/Bookworm running ffmpeg 5.1.4
It succeeds on:
- Ubuntu 18.04 - ffmpeg 3.4.11
- RPi4/Buster - ffmpeg 4.1.11
- Win11/Cygwin - ffmpeg 4.0.2
ON THE OTHER HAND, I tried some test HLS streams from the web (https://hls.somafm.com/) and they all seemed to work on RPi5/Bookworm.
So it seems like go2rtc is generating a flavor of HLS/fMP4 that is not recognized by the latest version of ffmpeg on the RPi5 Very weird indeed!
FFmpeg 5 is not a latest. It's very old version. Also you don't need -re flag for real time source. Also HLS is worst existing format for real time.
It fails with/without the 're' flag. Unfortunately, ffmpeg 5.1.4 is the version shipped with the latest Debian Bookworm (which itself was only released about 6 months ago) HLS may be the worst but unfortunately it seems to be the only format of mp4 that is supported out of the box by Octoprint webcams. But I then need to stream that to rtp which seems to be broken using go2rtc.
I REALLY LOVE YOUR GO2RTC PROGRAM -- AND I WOULD LIKE TO ENCOURAGE IT TO BE USED MORE BROADLY WITH OCTOPRINT. BUT I NEED TO GET IT WORKING WITH FFMPEG
@puterboy have you tried to use latest static ffmpeg? I had a similar issue on my raspberry pi.
Download static build from here https://johnvansickle.com/ffmpeg/, extract it, and use following block in go2rtc to override binary
ffmpeg:
bin: /tmp/ffmpeg-7.0.1-armel-static/ffmpeg
