frigate
frigate copied to clipboard
blakeblackshear
[HW Accel Support]: nvidia h265->h264 green screen, Reolink 820A, Nvidia Tesla P4
Describe the problem you are having
#video=h264 works without hardware acceleration but produces a green screen when enabled
Version
(0.12.1-367d724)
Frigate config file
mqtt:
enabled: False
ui:
live_mode: webrtc
go2rtc:
log:
exec: trace
# Enable webrtc
webrtc:
candidates:
- 10.1.2.2:8555
- stun:8555
streams:
nursery_265:
- "ffmpeg:rtsp://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}@10.1.2.12:554/h265Preview_01_main"
nursery_new: #transcoded
- "ffmpeg:nursery_265#video=h264#audio=opus#hardware"
ffmpeg:
output_args:
record: preset-record-generic-audio-aac
hwaccel_args: preset-nvidia-h264
cameras:
nursery:
live:
stream_name: nursery_264
ffmpeg:
inputs:
- path: rtsp://127.0.0.1:8554/nursery_sub_264?video=copy&audio=opus
input_args: preset-rtsp-restream
roles:
- detect
- path: rtsp://127.0.0.1:8554/nursery_265?video=copy&audio=copy
input_args: preset-rtsp-restream
roles:
- record
hwaccel_args: preset-nvidia-h264
record:
enabled: false
detect:
enabled: false
docker-compose file or Docker CLI command
version: "3.9"
services:
frigate:
container_name: frigate
deploy:
resources:
reservations:
devices:
- driver: nvidia
#device_ids: ['0'] # this is only needed when using multiple GPUs
count: 1 # number of GPUs
capabilities: [gpu]
privileged: true # this may not be necessary for all setups
restart: unless-stopped
image: ghcr.io/blakeblackshear/frigate:stable
shm_size: "150mb" # update for your cameras based on calculation above
devices:
- /dev/apex_0:/dev/apex_0 # passes a PCIe Coral, follow driver instructions here https://coral.ai/docs/m2/get-started/#2a-on-linux
- /dev/nvidia0:/dev/nvidia0
- /dev/nvidiactl:/dev/nvidiactl
- /dev/nvidiacaps:/dev/nvidiacaps
- /dev/nvidia-uvm:/dev/nvidia-uvm
- /dev/nvidia-uvm-tools:/dev/nvidia-uvm-tools
volumes:
- /etc/localtime:/etc/localtime:ro
#- /dev/apex_O:/dev/apex_O
- /root/frigate.yml:/config/config.yml
- /mnt/rust/storage:/media/frigate
ports:
- "5000:5000"
- "8554:8554" # RTSP feeds
- "8555:8555/tcp" # WebRTC over tcp
- "8555:8555/udp" # WebRTC over udp
environment:
FRIGATE_RTSP_USER: ${FRIGATE_RTSP_USER}
FRIGATE_RTSP_PASSWORD: ${FRIGATE_RTSP_PASSWORD}
FRIGATE_MQTT_USER: ${FRIGATE_MQTT_USER}
FRIGATE_MQTT_PASSWORD: ${FRIGATE_MQTT_PASSWORD}
Relevant log output
2024-01-27 14:16:43.100304164 14:16:43.100 TRC [exec] close url="exec:ffmpeg -hide_banner -allowed_media_types video+audio -fflags nobuffer -flags low_delay -timeout 5000000 -user_agent go2rtc/ffmpeg -rtsp_transport tcp -i rtsp://admin:[email protected]:554/h265Preview_01_main -c copy -user_agent ffmpeg/go2rtc -rtsp_transport tcp -f rtsp rtsp://localhost:8554/bad4129813a55f78f5cfbe1965c6ed72"
2024-01-27 14:16:43.267905070 frame= 15 fps=4.8 q=-1.0 size=N/A time=00:00:05.44 bitrate=N/A speed=1.75x
14:16:43.267 TRC [exec] close url="exec:ffmpeg -hide_banner -hwaccel cuda -hwaccel_output_format cuda -extra_hw_frames 2 -fflags nobuffer -flags low_delay -timeout 5000000 -user_agent go2rtc/ffmpeg -rtsp_transport tcp -i rtsp://localhost:8554/nursery_265?video&audio -c:v h264_nvenc -g 50 -profile:v high -level:v auto -preset:v p2 -tune:v ll -c:a libopus -ar:a 48000 -ac:a 2 -user_agent ffmpeg/go2rtc -rtsp_transport tcp -f rtsp rtsp://localhost:8554/b9540a7fbb36634cbe756b326026a9a8"
2024-01-27 14:16:43.896733531 size=N/A time=00:00:03.31 bitrate=N/A speed=2.53x
frame= 23 fps=6.3 q=-1.0 size=N/A time=00:00:06.08 bitrate=N/A speed=1.68x
Input #0, rtsp, from 'rtsp://admin:[email protected]:554/h265Preview_01_main':
2024-01-27 14:16:43.896742301 Metadata:
2024-01-27 14:16:43.896746336 title : Session streamed by "preview"
2024-01-27 14:16:43.896749813 comment : h265Preview_01_main
2024-01-27 14:16:43.896753287 Duration: N/A, start: 0.000000, bitrate: N/A
2024-01-27 14:16:43.896782174 Stream #0:0: Video: hevc (Main), yuv420p(tv), 3840x2160, 24.92 tbr, 90k tbn
2024-01-27 14:16:43.896798536 Stream #0:1: Audio: aac (LC), 16000 Hz, mono, fltp
2024-01-27 14:16:43.941419990 14:16:43.941 DBG [exec] run launch=955.052669ms
2024-01-27 14:16:43.941477409 Output #0, rtsp, to 'rtsp://localhost:8554/bad4129813a55f78f5cfbe1965c6ed72':
2024-01-27 14:16:43.941481230 Metadata:
2024-01-27 14:16:43.941497072 title : Session streamed by "preview"
2024-01-27 14:16:43.941500643 comment : h265Preview_01_main
2024-01-27 14:16:43.941515143 encoder : Lavf59.27.100
2024-01-27 14:16:43.941574524 Stream #0:0: Video: hevc (Main), yuv420p(tv), 3840x2160, q=2-31, 24.92 tbr, 90k tbn
2024-01-27 14:16:43.941611437 Stream #0:1: Audio: aac (LC), 16000 Hz, mono, fltp
2024-01-27 14:16:43.941632147 Stream mapping:
2024-01-27 14:16:43.941637006 Stream #0:0 -> #0:0 (copy)
2024-01-27 14:16:43.941641778 Stream #0:1 -> #0:1 (copy)
2024-01-27 14:16:43.941646741 Press [q] to stop, [?] for help
2024-01-27 14:16:45.343247715 frame= 0 fps=0.0 q=-1.0 size=N/A time=00:00:00.00 bitrate=N/A speed= 0x
frame= 31 fps=7.5 q=-1.0 size=N/A time=00:00:06.52 bitrate=N/A speed=1.58x
size=N/A time=00:00:03.95 bitrate=N/A speed=1.97x
frame= 4 fps=0.0 q=-1.0 size=N/A time=00:00:01.44 bitrate=N/A speed=2.85x
frame= 38 fps=8.2 q=-1.0 size=N/A time=00:00:07.10 bitrate=N/A speed=1.53x
frame= 18 fps= 18 q=-1.0 size=N/A time=00:00:02.00 bitrate=N/A speed=1.99x
size=N/A time=00:00:04.59 bitrate=N/A speed=1.71x
frame= 46 fps=8.9 q=-1.0 size=N/A time=00:00:07.55 bitrate=N/A speed=1.46x
Input #0, rtsp, from 'rtsp://localhost:8554/nursery_265?video&audio':
2024-01-27 14:16:45.343256288 Metadata:
2024-01-27 14:16:45.343260573 title : go2rtc/1.2.0
2024-01-27 14:16:45.343264421 Duration: N/A, start: 0.000000, bitrate: N/A
2024-01-27 14:16:45.343296962 Stream #0:0: Video: hevc (Main), yuv420p(tv), 3840x2160, 25 fps, 24.92 tbr, 90k tbn
2024-01-27 14:16:45.343300572 Stream #0:1: Audio: aac (LC), 16000 Hz, mono, fltp
2024-01-27 14:16:46.342212026 frame= 30 fps= 20 q=-1.0 size=N/A time=00:00:02.48 bitrate=N/A speed=1.63x
size=N/A time=00:00:05.23 bitrate=N/A speed= 1.6x
frame= 53 fps=9.3 q=-1.0 size=N/A time=00:00:08.06 bitrate=N/A speed=1.42x
frame= 43 fps= 21 q=-1.0 size=N/A time=00:00:03.16 bitrate=N/A speed=1.56x
size=N/A time=00:00:05.87 bitrate=N/A speed=1.52x
frame= 61 fps=9.9 q=-1.0 size=N/A time=00:00:08.57 bitrate=N/A speed=1.39x
Stream mapping:
2024-01-27 14:16:46.342221266 Stream #0:0 -> #0:0 (hevc (native) -> h264 (h264_nvenc))
2024-01-27 14:16:46.342225244 Stream #0:1 -> #0:1 (aac (native) -> opus (libopus))
2024-01-27 14:16:46.342228625 Press [q] to stop, [?] for help
2024-01-27 14:16:46.345027130 [libopus @ 0x55936c6fdac0] No bit rate set. Defaulting to 96000 bps.
2024-01-27 14:16:46.646513465 frame= 56 fps= 22 q=-1.0 size=N/A time=00:00:03.67 bitrate=N/A speed=1.44x
[hevc @ 0x55936c6de240] Could not find ref with POC 21
2024-01-27 14:16:46.781977073 size=N/A time=00:00:06.25 bitrate=N/A speed=1.43x
14:16:46.781 DBG [exec] run launch=3.80636942s
2024-01-27 14:16:46.782055791 Output #0, rtsp, to 'rtsp://localhost:8554/b9540a7fbb36634cbe756b326026a9a8':
2024-01-27 14:16:46.782060126 Metadata:
2024-01-27 14:16:46.782076097 title : go2rtc/1.2.0
2024-01-27 14:16:46.782091442 encoder : Lavf59.27.100
2024-01-27 14:16:46.782197010 Stream #0:0: Video: h264 (High), cuda(tv, progressive), 3840x2160, q=2-31, 2000 kb/s, 24.92 fps, 90k tbn
2024-01-27 14:16:46.782213105 Metadata:
2024-01-27 14:16:46.782216422 encoder : Lavc59.37.100 h264_nvenc
2024-01-27 14:16:46.782232100 Side data:
2024-01-27 14:16:46.782253985 cpb: bitrate max/min/avg: 0/0/2000000 buffer size: 4000000 vbv_delay: N/A
2024-01-27 14:16:46.782291542 Stream #0:1: Audio: opus, 48000 Hz, stereo, flt, 96 kb/s
2024-01-27 14:16:46.782296100 Metadata:
2024-01-27 14:16:46.782315059 encoder : Lavc59.37.100 libopus
2024-01-27 14:16:47.364723987 frame= 68 fps= 10 q=-1.0 size=N/A time=00:00:09.15 bitrate=N/A speed=1.37x
frame= 69 fps= 23 q=-1.0 size=N/A time=00:00:04.18 bitrate=N/A speed=1.37x
frame= 23 fps=0.0 q=39.0 size=N/A time=00:00:01.39 bitrate=N/A dup=22 drop=0 speed= 219x
frame= 24 fps=0.0 q=39.0 size=N/A time=00:00:01.39 bitrate=N/A dup=22 drop=0 speed=2.07x
size=N/A time=00:00:06.89 bitrate=N/A speed=1.39x
Impossible to convert between the formats supported by the filter 'Parsed_null_0' and the filter 'auto_scale_0'
2024-01-27 14:16:47.365099586 Error reinitializing filters!
2024-01-27 14:16:47.365105736 Failed to inject frame into filter network: Function not implemented
2024-01-27 14:16:47.365128039 Error while processing the decoded data for stream #0:0
2024-01-27 14:16:47.424324510 frame= 76 fps= 11 q=-1.0 size=N/A time=00:00:09.60 bitrate=N/A speed=1.33x
[libopus @ 0x55936c6fdac0] 1 frames left in the queue on closing
2024-01-27 14:16:47.429149167 av_interleaved_write_frame(): Broken pipe
2024-01-27 14:16:47.429309689 Error writing trailer of rtsp://localhost:8554/bad4129813a55f78f5cfbe1965c6ed72: Broken pipe
2024-01-27 14:16:47.429394313 frame= 80 fps= 23 q=-1.0 Lsize=N/A time=00:00:04.57 bitrate=N/A speed=1.31x
2024-01-27 14:16:47.429418337 video:2549kB audio:28kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
2024-01-27 14:16:47.430089375 Conversion failed!
2024-01-27 14:16:47.431505447 14:16:47.431 TRC [exec] close url="exec:ffmpeg -hide_banner -allowed_media_types video+audio -fflags nobuffer -flags low_delay -timeout 5000000 -user_agent go2rtc/ffmpeg -rtsp_transport tcp -i rtsp://admin:[email protected]:554/h265Preview_01_main -c copy -user_agent ffmpeg/go2rtc -rtsp_transport tcp -f rtsp rtsp://localhost:8554/bad4129813a55f78f5cfbe1965c6ed72"
2024-01-27 14:16:47.452245450 Conversion failed!
2024-01-27 14:16:47.463065128 14:16:47.462 WRN github.com/AlexxIT/go2rtc/cmd/streams/producer.go:133 > error=EOF url=ffmpeg:nursery_265#video=h264#audio=opus#hardware
2024-01-27 14:16:47.463210462 14:16:47.463 DBG [exec] run url="exec:ffmpeg -hide_banner -hwaccel cuda -hwaccel_output_format cuda -extra_hw_frames 2 -fflags nobuffer -flags low_delay -timeout 5000000 -user_agent go2rtc/ffmpeg -rtsp_transport tcp -i rtsp://localhost:8554/nursery_265?video&audio -c:v h264_nvenc -g 50 -profile:v high -level:v auto -preset:v p2 -tune:v ll -c:a libopus -ar:a 48000 -ac:a 2 -user_agent ffmpeg/go2rtc -rtsp_transport tcp -f rtsp rtsp://localhost:8554/b9540a7fbb36634cbe756b326026a9a8"
2024-01-27 14:16:47.473902139 14:16:47.473 DBG [exec] run url="exec:ffmpeg -hide_banner -allowed_media_types video+audio -fflags nobuffer -flags low_delay -timeout 5000000 -user_agent go2rtc/ffmpeg -rtsp_transport tcp -i rtsp://admin:[email protected]:554/h265Preview_01_main -c copy -user_agent ffmpeg/go2rtc -rtsp_transport tcp -f rtsp rtsp://localhost:8554/bad4129813a55f78f5cfbe1965c6ed72"
2024-01-27 14:16:47.511169313 14:16:47.510 TRC [exec] close url="exec:ffmpeg -hide_banner -hwaccel cuda -hwaccel_output_format cuda -extra_hw_frames 2 -fflags nobuffer -flags low_delay -timeout 5000000 -user_agent go2rtc/ffmpeg -rtsp_transport tcp -i rtsp://localhost:8554/nursery_265?video&audio -c:v h264_nvenc -g 50 -profile:v high -level:v auto -preset:v p2 -tune:v ll -c:a libopus -ar:a 48000 -ac:a 2 -user_agent ffmpeg/go2rtc -rtsp_transport tcp -f rtsp rtsp://localhost:8554/b9540a7fbb36634cbe756b326026a9a8"
2024-01-27 14:16:49.121273691 size=N/A time=00:00:07.39 bitrate=N/A speed=1.35x
frame= 83 fps= 11 q=-1.0 size=N/A time=00:00:10.17 bitrate=N/A speed=1.32x
frame= 91 fps= 11 q=-1.0 size=N/A time=00:00:10.62 bitrate=N/A speed=1.29x
size=N/A time=00:00:08.03 bitrate=N/A speed= 1.3x
frame= 98 fps= 11 q=-1.0 size=N/A time=00:00:11.20 bitrate=N/A speed=1.28x
size=N/A time=00:00:08.67 bitrate=N/A speed=1.28x
Input #0, rtsp, from 'rtsp://admin:[email protected]:554/h265Preview_01_main':
2024-01-27 14:16:49.121287140 Metadata:
2024-01-27 14:16:49.121291251 title : Session streamed by "preview"
2024-01-27 14:16:49.121319360 comment : h265Preview_01_main
2024-01-27 14:16:49.121323008 Duration: N/A, start: 0.000000, bitrate: N/A
2024-01-27 14:16:49.121362430 Stream #0:0: Video: hevc (Main), yuv420p(tv), 3840x2160, 25 fps, 24.92 tbr, 90k tbn
2024-01-27 14:16:49.121397597 Stream #0:1: Audio: aac (LC), 16000 Hz, mono, fltp
2024-01-27 14:16:49.165412757 14:16:49.165 DBG [exec] run launch=1.691438117s
2024-01-27 14:16:49.165445815 Output #0, rtsp, to 'rtsp://localhost:8554/bad4129813a55f78f5cfbe1965c6ed72':
2024-01-27 14:16:49.165449483 Metadata:
2024-01-27 14:16:49.165453260 title : Session streamed by "preview"
2024-01-27 14:16:49.165456584 comment : h265Preview_01_main
2024-01-27 14:16:49.165471375 encoder : Lavf59.27.100
2024-01-27 14:16:49.165519296 Stream #0:0: Video: hevc (Main), yuv420p(tv), 3840x2160, q=2-31, 25 fps, 24.92 tbr, 90k tbn
2024-01-27 14:16:49.165558325 Stream #0:1: Audio: aac (LC), 16000 Hz, mono, fltp
2024-01-27 14:16:49.165561505 Stream mapping:
2024-01-27 14:16:49.165576913 Stream #0:0 -> #0:0 (copy)
2024-01-27 14:16:49.165579983 Stream #0:1 -> #0:1 (copy)
2024-01-27 14:16:49.165583020 Press [q] to stop, [?] for help
FFprobe output from your camera
[{"return_code":0,"stderr":"","stdout":{"programs":[],"streams":[{"avg_frame_rate":"15/1","codec_long_name":"H.264/AVC/MPEG-4AVC/MPEG-4part10","height":360,"width":640},{"avg_frame_rate":"0/0","codec_long_name":"Opus(OpusInteractiveAudioCodec)"}]}},{"return_code":0,"stderr":"","stdout":{"programs":[],"streams":[{"avg_frame_rate":"0/0","codec_long_name":"AAC(AdvancedAudioCoding)"},{"avg_frame_rate":"25/1","codec_long_name":"H.265/HEVC(HighEfficiencyVideoCoding)","height":2160,"width":3840}]}}]
Operating system
Proxmox
Install method
Docker Compose
Network connection
Wired
Camera make and model
Reolink RLC-820A
Any other information that may be helpful
I've tested that hardware acceleration works in general. The Proxmox host shows processes running ffmpeg in nvtop, and the System tab in Frigate shows little CPU and some GPU when it is running.
When #video=h264#hardware is added, the result is a split green screen, half light green half dark green
When #hardware is removed, the stream is successfully transcoded, but CPU use is very high. This makes me think the h265 stream is valid and not the problem. The raw h265 stream also plays fine in VLC.
The GPU used is the Tesla P4.
I have tried a variety of ffmpeg manual parameters and overrides with no success. On the h264 stream, I'm able to successfully resize and perform other transformations using hardware acceleration.
I've tried updating go2rtc to the latest version as well (1.8.5).
I can't remember if I get the same results with the Reolink Duo 2, will check if needed
Nvidia drivers are at 535.129.03
~Strangely, I got this to work when I changed the streams section to~
nursery_new: #transcoded
- "ffmpeg:rtsp://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}@10.1.2.12:554/h265Preview_01_main"
- "ffmpeg:nursery_new#video=h264#audio=opus#hardware"
~Was I just misunderstanding how referencing another go2rtc entry works? but then, why did it work the way I had originally with CPU but not GPU?~
Edit: it's not working, I was just getting a valid stream to play for the first time in a while, but it was the h265 stream, not the transcoded h264 stream
@NickM-27 I see what I did. After my update, I am just getting the raw h265 stream (I was wondering why it was suddenly MSE instead of rtc...), not the transcoded one, so it's still not actually working ☹️
When I use
nursery_new:
- "ffmpeg:rtsp://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}@10.1.2.12:554/h265Preview_01_main"
- "ffmpeg:nursery_new#video=h264#audio=opus#hardware"
I get
{
"producers": [
{
"type": "RTSP source",
"url": "rtsp://localhost:8554/af8bc1e3122aa2942761f052d00aa539",
"remote_addr": "127.0.0.1:37396",
"user_agent": "ffmpeg/go2rtc",
"medias": [
"video, sendonly, 96 H265/90000",
"audio, sendonly, 97 MPEG4-GENERIC/16000"
],
"tracks": [
"96 H265/90000, sinks=1",
"97 MPEG4-GENERIC/16000, sinks=1"
],
"recv": 116268881
},
{
"url": "ffmpeg:nursery_new#video=h264#audio=opus#hardware"
}
],
"consumers": [
{
"type": "MP4 client",
"remote_addr": "127.0.0.1:46876",
"user_agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36",
"send": 34164343
}
]
}
because it's publishing both as options, so I guess it negotiates for the h265 version?
but when I use
nursery_new:
- "ffmpeg:rtsp://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}@10.1.2.12:554/h265Preview_01_main#video=h264#audio=opus#hardware"
then I get the green screen and the following go2rtc info
{
"producers": [
{
"type": "RTSP source",
"url": "rtsp://localhost:8554/4343896967c3ac03add6ba88ba976a6a",
"remote_addr": "127.0.0.1:49066",
"user_agent": "ffmpeg/go2rtc",
"medias": [
"video, sendonly, 96 H264/90000",
"audio, sendonly, 97 OPUS/48000/2"
],
"tracks": [
"96 H264/90000, sinks=1",
"97 OPUS/48000/2, sinks=1"
],
"recv": 107081
}
],
"consumers": [
{
"type": "WebRTC client",
"remote_addr": "udp4 prflx 192.168.82.27:56536 related :0",
"user_agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36",
"send": 568798
}
]
}
Your cameras stream is h264 not h265 according to the info
Yes, exactly. I'm attempting to transcode from the underlying h265 stream into h264 (and eventually do some additional transformations), and stream it all with webrtc. The problem is, whenever I involve hardware for this transcoding, I get the split green screen.
Really? The things that made me think it is h265 are the 2nd entry in the ffprobe data
[
...,
{
"return_code": 0,
"stderr": "",
"stdout": {
"programs": [],
"streams": [
{
"avg_frame_rate": "0/0",
"codec_long_name": "AAC(AdvancedAudioCoding)"
},
{
"avg_frame_rate": "25/1",
"codec_long_name": "H.265/HEVC(HighEfficiencyVideoCoding)",
"height": 2160,
"width": 3840
}
]
}
}
]
and when I use
nursery_265:
- "ffmpeg:rtsp://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}@10.1.2.12:554/h265Preview_01_main"
the go2rtc info looks like
{
"producers": [
{
"type": "RTSP source",
"url": "rtsp://localhost:8554/af8bc1e3122aa2942761f052d00aa539",
"remote_addr": "127.0.0.1:54276",
"user_agent": "ffmpeg/go2rtc",
"medias": [
"video, sendonly, 96 H265/90000",
"audio, sendonly, 97 MPEG4-GENERIC/16000"
],
"tracks": [
"96 H265/90000, sinks=2",
"97 MPEG4-GENERIC/16000, sinks=2"
],
"recv": 131627999
}
],
"consumers": [
{
"type": "RTSP client",
"url": "rtsp://127.0.0.1:8554/nursery_265?video=copy\u0026audio=copy",
"remote_addr": "127.0.0.1:54266",
"user_agent": "FFmpeg Frigate/0.12.1-367d724",
"medias": [
"video, recvonly, 96 H265/90000",
"audio, recvonly, 97 MPEG4-GENERIC/16000"
],
"tracks": [
"96 H265/90000, sinks=2",
"97 MPEG4-GENERIC/16000, sinks=2"
],
"send": 131626151
},
{
"type": "MP4 client",
"remote_addr": "127.0.0.1:42718",
"user_agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36",
"send": 44635563
}
]
}
don't "codec_long_name": "H.265/HEVC(HighEfficiencyVideoCoding)", and "video, sendonly, 96 H265/90000", mean the camera produces a h265 stream? or what am I still missing?
Either way, adding #video=h264 is the part that works with cpu but not gpu when I add #hardware
oh I thought I was looking at go2rtc but it was ffrpobe. Okay, you most likely need to enable trace logs for exec and see what it says
go2rtc:
log:
exec: trace
Hi, I also just had this issue. The relevant debug log output is here:
frigate | 2024-02-12 01:12:41.455899602 01:12:41.455 DBG [exec] run url="exec:ffmpeg -hide_banner -hwaccel cuda -hwaccel_output_format cuda -allowed_media_types video -fflags nobuffer -flags low_delay -timeout 5000000 -user_agent go2rtc/ffmpeg -rtsp_flags prefer_tcp -i rtsp://frigate:[email protected]:554/h265Preview_01_main -c:v h264_nvenc -g 50 -bf 0 -profile:v high -level:v auto -preset:v p2 -tune:v ll -an -user_agent ffmpeg/go2rtc -rtsp_transport tcp -f rtsp {output}"
frigate | 2024-02-12 01:12:43.287231191 Input #0, rtsp, from 'rtsp://frigate:[email protected]:554/h265Preview_01_main':
frigate | 2024-02-12 01:12:43.287241426 Metadata:
frigate | 2024-02-12 01:12:43.287247814 title : Session streamed by "preview"
frigate | 2024-02-12 01:12:43.287253326 comment : h265Preview_01_main
frigate | 2024-02-12 01:12:43.287303992 Duration: N/A, start: 0.040078, bitrate: N/A
frigate | 2024-02-12 01:12:43.287422727 Stream #0:0: Video: hevc (Main), yuv420p(tv), 3840x2160, 25 fps, 24.92 tbr, 90k tbn
frigate | 2024-02-12 01:12:43.361830381 Stream mapping:
frigate | 2024-02-12 01:12:43.361833163 Stream #0:0 -> #0:0 (hevc (native) -> h264 (h264_nvenc))
frigate | 2024-02-12 01:12:43.361834163 Press [q] to stop, [?] for help
frigate | 2024-02-12 01:12:43.381068647 [hevc @ 0x627f0d036440] Could not find ref with POC 21
frigate | 2024-02-12 01:12:43.456558711 01:12:43.456 DBG [exec] run launch=2.000649193s
frigate | 2024-02-12 01:12:43.456658780 Output #0, rtsp, to 'rtsp://127.0.0.1:8554/4b5c69a7d62be7b67b5c296dc17c88f9':
frigate | 2024-02-12 01:12:43.456662927 Metadata:
frigate | 2024-02-12 01:12:43.456680201 title : Session streamed by "preview"
frigate | 2024-02-12 01:12:43.456683205 comment : h265Preview_01_main
frigate | 2024-02-12 01:12:43.456685502 encoder : Lavf59.27.100
frigate | 2024-02-12 01:12:43.456758157 Stream #0:0: Video: h264 (High), cuda(tv, progressive), 3840x2160, q=2-31, 2000 kb/s, 24.92 fps, 90k tbn
frigate | 2024-02-12 01:12:43.456761620 Metadata:
frigate | 2024-02-12 01:12:43.456764395 encoder : Lavc59.37.100 h264_nvenc
frigate | 2024-02-12 01:12:43.456766556 Side data:
frigate | 2024-02-12 01:12:43.456769033 cpb: bitrate max/min/avg: 0/0/2000000 buffer size: 4000000 vbv_delay: N/A
Impossible to convert between the formats supported by the filter 'Parsed_null_0' and the filter 'auto_scale_0'ed=N/A
frigate | 2024-02-12 01:12:44.677010255 Error reinitializing filters!
frigate | 2024-02-12 01:12:44.677012709 Failed to inject frame into filter network: Function not implemented
frigate | 2024-02-12 01:12:44.677023854 Error while processing the decoded data for stream #0:0
frigate | 2024-02-12 01:12:44.718669183 Conversion failed!
From testing, I found the issue is with the -fflags nobuffer flag.
Removing that flag fixes it:
go2rtc:
ffmpeg:
rtsp: "-flags low_delay -timeout 5000000 -user_agent go2rtc/ffmpeg -rtsp_flags prefer_tcp -i {input}"
streams:
side:
- ffmpeg:rtsp://frigate:[email protected]:554/h265Preview_01_main#video=h264#hardware
The rtsp flags used are the default ones[1], just without -fflags nobuffer.
[1]: https://github.com/AlexxIT/go2rtc/blob/5fa31fe4d6cf0e77562b755d52e8ed0165f89d25/internal/ffmpeg/ffmpeg.go#L44
Hope this helps
The rtsp flags used are the default ones[1], just without -fflags nobuffer.
Wow, this is great, removing -fflags nobuffer got it to work. How did you even find this?
I'm currently removing this flag by adding the following config. (There's probably a better way, but I didn't understand the rtsp section you used above)
ffmpeg:
hw_input: >-
-hwaccel cuda
-hwaccel_output_format cuda
-extra_hw_frames 2
-flags low_delay
-timeout 5000000
-user_agent go2rtc/ffmpeg
-rtsp_transport tcp
-i {input}\
hw_h264: >-
-c:v h264_nvenc
-g 50
-profile:v high
-level:v auto
-preset:v p2
-tune:v ll
and updating my transcoding stream to
nursery_new: #transcoded
- "ffmpeg:nursery_265#input=hw_input#video=hw_h264#audio=opus#hardware"
Upon a 2nd look, this doesn't seem to scale. Once I apply this to a few cameras at once, things start going a bit haywire, including at least one stream with a green screen. Maybe I'm still doing something wrong, will look again with fresh eyes later
Here are two sets of logs with exec: trace. they are both from a config where -fflags nobuffer is removed. I changed literally nothing in the config between these two runs, just saved and restarted Frigate. The only real difference I can see so far is the timing of when the transcoded stream started.
no-green-screen-logs.txt green-screen-logs.txt
The config used in both cases included
nursery_original: #for recording
- "ffmpeg:rtsp://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}@10.1.2.12:554/h265Preview_01_main"
nursery_balanced: #transcoded for live preview
- "ffmpeg:nursery_original#input=hw_input#video=hw_h264#raw=-vf scale_cuda=1024:-1#audio=opus#hardware"
I hope this helps narrow it down
the hardware filter doesn't like the scaling, you could try doing a manual exec call and using -vf hwupload,scale_cuda=640:-1,hwdownload
Like this?
nursery_balanced: exec:ffmpeg -hide_banner -hwaccel cuda -hwaccel_output_format cuda -extra_hw_frames 2 -flags low_delay -timeout 5000000 -user_agent go2rtc/ffmpeg -rtsp_transport tcp -i rtsp://localhost:8554/nursery_original?video&audio -vf "hwupload,scale_cuda=1024:-1,hwdownload" -c:v h264_nvenc -g 50 -profile:v high -level:v auto -preset:v p2 -tune:v ll -c:a libopus -ar:a 48000 -ac:a 2 -user_agent ffmpeg/go2rtc -rtsp_transport tcp -f rtsp {{output}}
I got the following when I tried that
2024-02-13 17:31:38.602119766 frame= 342 fps= 23 q=-1.0 size=N/A time=00:00:15.93 bitrate=N/A speed=1.07x
frame= 355 fps= 23 q=-1.0 size=N/A time=00:00:16.44 bitrate=N/A speed=1.07x
Input #0, rtsp, from 'rtsp://localhost:8554/nursery_original?video&audio':
2024-02-13 17:31:38.602127968 Metadata:
2024-02-13 17:31:38.602131890 title : go2rtc/1.2.0
2024-02-13 17:31:38.602135417 Duration: N/A, start: 0.000000, bitrate: N/A
2024-02-13 17:31:38.602168048 Stream #0:0: Video: hevc (Main), yuv420p(tv), 3840x2160, 25 fps, 24.92 tbr, 90k tbn
2024-02-13 17:31:38.602183062 Stream #0:1: Audio: aac (LC), 16000 Hz, mono, fltp
2024-02-13 17:31:38.830215607 Stream mapping:
2024-02-13 17:31:38.830228358 Stream #0:0 -> #0:0 (hevc (native) -> h264 (h264_nvenc))
2024-02-13 17:31:38.830234558 Stream #0:1 -> #0:1 (aac (native) -> opus (libopus))
2024-02-13 17:31:38.830267893 Press [q] to stop, [?] for help
2024-02-13 17:31:38.888238287 [hevc @ 0x564e0f7c2440] Could not find ref with POC 32
2024-02-13 17:31:38.941734616 [hwdownload @ 0x564e10c626c0] Invalid output format yuv420p for hwframe download.
2024-02-13 17:31:38.941743699 [Parsed_hwdownload_2 @ 0x564e10c625c0] Failed to configure output pad on Parsed_hwdownload_2
2024-02-13 17:31:38.943368404 Error reinitializing filters!
2024-02-13 17:31:38.943373971 Failed to inject frame into filter network: Invalid argument
2024-02-13 17:31:38.943395776 Error while processing the decoded data for stream #0:0
2024-02-13 17:31:38.974658522 Conversion failed!
2024-02-13 17:31:39.040272431 17:31:39.040 ERR github.com/AlexxIT/go2rtc/cmd/streams/producer.go:53 > error="exec: exit status 1" url="exec:ffmpeg -hide_banner -hwaccel cuda -hwaccel_output_format cuda -extra_hw_frames 2 -flags low_delay -timeout 5000000 -user_agent go2rtc/ffmpeg -rtsp_transport tcp -i rtsp://localhost:8554/nursery_original?video&audio -vf \"hwupload,scale_cuda=1024:-1,hwdownload\" -c:v h264_nvenc -g 50 -profile:v high -level:v auto -preset:v p2 -tune:v ll -c:a libopus -ar:a 48000 -ac:a 2 -user_agent ffmpeg/go2rtc -rtsp_transport tcp -f rtsp {output}"
2024-02-13 17:31:39.040369476 17:31:39.040 ERR github.com/AlexxIT/go2rtc/cmd/mjpeg/mjpeg.go:159 > error="source 0 error: exec: exit status 1"
I've done some additional testing with the earlier config, and there are no green screens if I have webrtc pages open for each camera when Frigate starts up (trying to reproduce the timing hinted at by the logs where there was no green screen). But if I start the streams any time afterwards, they're just green. What could this mean? And how might the nobuffer flag be affecting things? Are there 2 separate issues here?
try this
nursery_balanced: exec:ffmpeg -hide_banner -hwaccel cuda -hwaccel_output_format cuda -extra_hw_frames 2 -flags low_delay -timeout 5000000 -user_agent go2rtc/ffmpeg -rtsp_transport tcp -i rtsp://localhost:8554/nursery_original?video&audio -vf "scale_cuda=w={1}:h={2}:format=nv12,hwdownload,format=nv12,format=yuv420p" -c:v h264_nvenc -g 50 -profile:v high -level:v auto -preset:v p2 -tune:v ll -c:a libopus -ar:a 48000 -ac:a 2 -user_agent ffmpeg/go2rtc -rtsp_transport tcp -f rtsp {{output}}
the logs indicate ffmpeg failed because the nvidia filter was not happy
this seems to result in Frigate crashing
[INFO] Preparing go2rtc config...
[INFO] Not injecting WebRTC candidates into go2rtc config as it has been set manually
[INFO] The go2rtc service exited with code 1 (by signal 0)
I'm happy to leave downscaling out of it for now if we want to simplify the scope of this issue
edit: doh, I didn't realize I had to replace {1} and {2} in scale_cuda=w={1}:h={2}
For this and another stream without downscaling, it started out green, but right before I closed the window it cleared up. Doing some more experimenting and log collection now
I have done more testing and I think the nobuffer setting was a red herring.
I think the problem is that autoscaling is being applied by ffmpeg, I assume because there are some weird frames coming from reolink.
Try your original config from https://github.com/blakeblackshear/frigate/issues/9469#issuecomment-1940216914 but with the -noautoscale flag
ffmpeg:
hw_input: >-
-hwaccel cuda
-hwaccel_output_format cuda
-extra_hw_frames 2
-flags low_delay
-timeout 5000000
-user_agent go2rtc/ffmpeg
-rtsp_transport tcp
-i {input}\
hw_h264: >-
-c:v h264_nvenc
-g 50
-profile:v high
-level:v auto
-preset:v p2
-tune:v ll
-noautoscale
EDIT: Can probably readd the -fflags nobuffer to hw_input also
I am getting fairly consistently decent results by adding the -noautoscale flag. Same with the long manual exec command. I'm not sure which one is better or why, and I only have vague qualitative feedback at this point.
In both cases, of my 4 streams, 2 start as RTC and 2 as MSE. At a certain point, the MSE streams switch to RTC but then freeze. I'm not sure this is related to any of the above. Also possibly related, my CPU appears very busy, with ~80% for python3 and ~70% for frigate.output` (no birdseye, no detection yet). Which part of this is CPU heavy? If these are off-topic, feel free to ignore them
Edit: the cpu issue seems to have resolved itself (?). the MSE -> RTC freeze issue remains, but I could understand if that's a separate Reolink thing or some other issue
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}