frigate-hass-addons
frigate-hass-addons copied to clipboard
Trying to enable hardware acceleration on GPU (Intel 11th gen) just get green screen
I'm wonering if anyone has this working or if this is even possible.
I have the Hassos addon running inside the lastest version of HassOS.
I recently upgraded my hardware and HassOS runs directly on the hardware which is a 11th Gen Intel(R) Core(TM) i5-11400T.
I use a coral device for object detection which is working fine. I don't know if using the hardware acceleration is possible with the addon vs running frigate in a dedicated docker container as I have seen else where ?
I have had the camera's running for a long time and all work well but ffmpeg chews quite a bit of CPU and I wanted to enable the hardware acceleration support to reduce this.
I have checked the addon by connecting to the docker with a bash shell and I can see the /dev/dri/renderD128 device is available.
When i try to enable the hw acceleration I just get a green screen on each camera. Disable hwacceleration and it works all be it with high CPU. I don't seem to see any information about ffmpeg using the hardware for processing however.
frigate.yaml (ffmpeg)
ffmpeg:
# Optional: global ffmpeg args (default: shown below)
#global_args: -hide_banner -loglevel warning
global_args: -hide_banner -loglevel verbose
#hwaccel_args: []
# QSV Settings
hwaccel_args:
- -hwaccel
- qsv
- -hwaccel_device
- /dev/dri/renderD128
- -hwaccel_output_format
- yuv420p
- -c:v
- h264_qsv
# VAAPI Settings
# hwaccel_args:
# - -hwaccel
# - vaapi
# - -hwaccel_device
# - /dev/dri/renderD128
# - -hwaccel_output_format
# - vaapi
# Alternate VAAPI Settings
# hwaccel_args:
# - -init_hw_device
# - vaapi=intel:/dev/dri/renderD128
# - -hwaccel
# - vaapi
# - -hwaccel_device
# - intel
# - -hwaccel_output_format
# - yuv420p
input_args: -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1
# Optional: global output args
output_args:
detect: -f rawvideo -pix_fmt yuv420p
# Optional: output args for record streams (default: shown below)
record: -f segment -segment_time 60 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -an
# Optional: output args for clips streams (default: shown below)
#clips: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -an
# Optional: output args for rtmp streams (default: shown below)
rtmp: -c copy -f flv
Startup log:
[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] done.
[services.d] starting services
[services.d] done.
[2022-10-04 11:38:19] frigate.app INFO : Starting Frigate (0.11.0-3846a13)
[2022-10-04 11:38:19] frigate.app INFO : Creating directory: /tmp/cache
Starting migrations
[2022-10-04 11:38:19] peewee_migrate INFO : Starting migrations
There is nothing to migrate
[2022-10-04 11:38:19] peewee_migrate INFO : There is nothing to migrate
[2022-10-04 11:38:19] detector.coral INFO : Starting detection process: 232
[2022-10-04 11:38:19] frigate.app INFO : Output process started: 233
[2022-10-04 11:38:19] ws4py INFO : Using epoll
[2022-10-04 11:38:19] frigate.edgetpu INFO : Attempting to load TPU as usb
[2022-10-04 11:38:19] frigate.app INFO : Camera processor started for living_room: 239
[2022-10-04 11:38:22] frigate.edgetpu INFO : TPU found
[2022-10-04 11:38:19] frigate.app INFO : Camera processor started for study: 241
[2022-10-04 11:38:19] frigate.app INFO : Camera processor started for kids_room: 245
[2022-10-04 11:38:19] frigate.app INFO : Camera processor started for front_door: 247
[2022-10-04 11:38:19] frigate.app INFO : Camera processor started for back_garden: 249
[2022-10-04 11:38:19] frigate.app INFO : Camera processor started for kitchen: 250
[2022-10-04 11:38:19] frigate.app INFO : Camera processor started for parking: 252
[2022-10-04 11:38:19] frigate.app INFO : Capture process started for living_room: 253
[2022-10-04 11:38:19] frigate.app INFO : Capture process started for study: 255
[2022-10-04 11:38:19] frigate.app INFO : Capture process started for kids_room: 259
[2022-10-04 11:38:19] frigate.app INFO : Capture process started for front_door: 264
[2022-10-04 11:38:19] frigate.app INFO : Capture process started for back_garden: 284
[2022-10-04 11:38:19] frigate.app INFO : Capture process started for kitchen: 292
[2022-10-04 11:38:19] frigate.app INFO : Capture process started for parking: 301
[2022-10-04 11:38:19] ws4py INFO : Using epoll
[2022-10-04 11:38:39] watchdog.living_room INFO : No frames received from living_room in 20 seconds. Exiting ffmpeg...
[2022-10-04 11:38:39] watchdog.living_room INFO : Waiting for ffmpeg to exit gracefully...
[2022-10-04 11:38:39] watchdog.study INFO : No frames received from study in 20 seconds. Exiting ffmpeg...
[2022-10-04 11:38:39] watchdog.study INFO : Waiting for ffmpeg to exit gracefully...
[2022-10-04 11:38:39] watchdog.kids_room INFO : No frames received from kids_room in 20 seconds. Exiting ffmpeg...
[2022-10-04 11:38:39] watchdog.kids_room INFO : Waiting for ffmpeg to exit gracefully...
[2022-10-04 11:38:39] watchdog.front_door INFO : No frames received from front_door in 20 seconds. Exiting ffmpeg...
[2022-10-04 11:38:39] watchdog.front_door INFO : Waiting for ffmpeg to exit gracefully...
[2022-10-04 11:38:39] watchdog.kitchen INFO : No frames received from kitchen in 20 seconds. Exiting ffmpeg...
[2022-10-04 11:38:39] watchdog.back_garden INFO : No frames received from back_garden in 20 seconds. Exiting ffmpeg...
[2022-10-04 11:38:39] watchdog.kitchen INFO : Waiting for ffmpeg to exit gracefully...
[2022-10-04 11:38:39] watchdog.back_garden INFO : Waiting for ffmpeg to exit gracefully...
[2022-10-04 11:38:39] frigate.video ERROR : kids_room: Unable to read frames from ffmpeg process.
[2022-10-04 11:38:39] frigate.video ERROR : kids_room: ffmpeg process is not running. exiting capture thread...
[2022-10-04 11:38:39] frigate.video ERROR : front_door: Unable to read frames from ffmpeg process.
[2022-10-04 11:38:39] frigate.video ERROR : front_door: ffmpeg process is not running. exiting capture thread...
[2022-10-04 11:38:39] frigate.video ERROR : back_garden: Unable to read frames from ffmpeg process.
[2022-10-04 11:38:39] frigate.video ERROR : back_garden: ffmpeg process is not running. exiting capture thread...
[2022-10-04 11:38:39] watchdog.parking INFO : No frames received from parking in 20 seconds. Exiting ffmpeg...
[2022-10-04 11:38:39] watchdog.parking INFO : Waiting for ffmpeg to exit gracefully...
[2022-10-04 11:38:39] frigate.video ERROR : kitchen: Unable to read frames from ffmpeg process.
[2022-10-04 11:38:39] frigate.video ERROR : kitchen: ffmpeg process is not running. exiting capture thread...
[2022-10-04 11:38:39] frigate.video ERROR : study: Unable to read frames from ffmpeg process.
[2022-10-04 11:38:39] frigate.video ERROR : study: ffmpeg process is not running. exiting capture thread...
[2022-10-04 11:38:39] frigate.video ERROR : living_room: Unable to read frames from ffmpeg process.
[2022-10-04 11:38:39] frigate.video ERROR : living_room: ffmpeg process is not running. exiting capture thread...
[2022-10-04 11:38:39] frigate.video ERROR : parking: Unable to read frames from ffmpeg process.
[2022-10-04 11:38:39] frigate.video ERROR : parking: ffmpeg process is not running. exiting capture thread...
If I start ffmpeg manually just changing the cache name of the camera e.g. the frigate ffmpeg is showing as the process:
ffmpeg -hide_banner -loglevel verbose -hwaccel qsv -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p -c:v h264_qsv -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsp://REDACTED:[email protected]:554/Streaming/Channels/101 -f segment -segment_time 60 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -an /tmp/cache/back_garden-%Y%m%d%H%M%S.mp4 -c copy -f flv rtmp://127.0.0.1/live/back_garden
If I use docker exec -it bash on the container for frigate and run the above command just changing the name for cache and rtmp as below
ffmpeg -hide_banner -loglevel verbose -hwaccel qsv -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p -c:v h264_qsv -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsp://REDACTED:[email protected]:554/Streaming/Channels/101 -f segment -segment_time 60 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -an /tmp/cache/test1-%Y%m%d%H%M%S.mp4 -c copy -f flv rtmp://127.0.0.1/live/test1
I get the following:
root@ccab4aaf-frigate:/opt/frigate# ffmpeg -hide_banner -loglevel verbose -hwaccel qsv -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p -c:v h264_qsv -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsp://REDACTED:[email protected]:554/Streaming/Channels/101 -f segment -segment_time 60 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -an /tmp/cache/test1-%Y%m%d%H%M%S.mp4 -c copy -f flv rtmp://127.0.0.1/live/test1
[tcp @ 0x55d359d5c280] Starting connection attempt to 192.168.1.84 port 554
[tcp @ 0x55d359d5c280] Successfully connected to 192.168.1.84 port 554
[rtsp @ 0x55d359d58fc0] SDP:
v=0
o=- 1664880513970817 1664880513970817 IN IP4 192.168.1.84
s=Media Presentation
e=NONE
b=AS:5100
t=0 0
a=control:rtsp://192.168.1.84:554/Streaming/Channels/101/
m=video 0 RTP/AVP 96
b=AS:5000
a=control:rtsp://192.168.1.84:554/Streaming/Channels/101/trackID=1
a=rtpmap:96 H264/90000
a=fmtp:96 profile-level-id=420029; packetization-mode=1; sprop-parameter-sets=Z00AKY2NQDwBE/LCAAARlAACvyAI,aO44gA==
m=audio 0 RTP/AVP 104
b=AS:50
a=control:rtsp://192.168.1.84:554/Streaming/Channels/101/trackID=2
a=rtpmap:104 mpeg4-generic/16000/1
a=fmtp:104 profile-level-id=15; streamtype=5; mode=AAC-hbr; config=1408;SizeLength=13; IndexLength=3; IndexDeltaLength=3; Profile=1;
a=Media_header:MEDIAINFO=494D4B48010100000400010001200110803E0000007D000000000000000000000000000000000000;
a=appversion:1.0
[rtsp @ 0x55d359d58fc0] setting jitter buffer size to 0
Last message repeated 1 times
[h264 @ 0x55d359d5e400] Reinit context to 1920x1088, pix_fmt: yuv420p
Input #0, rtsp, from 'rtsp://REDACTED:[email protected]:554/Streaming/Channels/101':
Metadata:
title : Media Presentation
Duration: N/A, start: 1664880515.383563, bitrate: N/A
Stream #0:0: Video: h264 (Main), 1 reference frame, yuv420p(progressive, left), 1920x1080 (1920x1088), 20 fps, 20 tbr, 90k tbn
Stream #0:1: Audio: aac (LC), 16000 Hz, mono, fltp
[tcp @ 0x55d359d78940] Starting connection attempt to 127.0.0.1 port 1935
[tcp @ 0x55d359d78940] Successfully connected to 127.0.0.1 port 1935
[segment @ 0x55d359d75080] Selected stream id:0 type:video
[segment @ 0x55d359d75080] Opening '/tmp/cache/test1-20221004114836.mp4' for writing
Output #0, segment, to '/tmp/cache/test1-%Y%m%d%H%M%S.mp4':
Metadata:
title : Media Presentation
encoder : Lavf59.27.100
Stream #0:0: Video: h264 (Main), 1 reference frame, yuv420p(progressive, left), 1920x1080 (0x0), q=2-31, 20 fps, 20 tbr, 10240 tbn
Output #1, flv, to 'rtmp://127.0.0.1/live/test1':
Metadata:
title : Media Presentation
encoder : Lavf59.27.100
Stream #1:0: Video: h264 (Main), 1 reference frame ([7][0][0][0] / 0x0007), yuv420p(progressive, left), 1920x1080 (0x0), q=2-31, 20 fps, 20 tbr, 1k tbn
Stream #1:1: Audio: aac (LC) ([10][0][0][0] / 0x000A), 16000 Hz, mono, fltp
Stream mapping:
Stream #0:0 -> #0:0 (copy)
Stream #0:0 -> #1:0 (copy)
Stream #0:1 -> #1:1 (copy)
Press [q] to stop, [?] for help
[segment @ 0x55d359d75080] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
[segment @ 0x55d359d75080] segment:'/tmp/cache/test1-20221004114836.mp4' starts with packet stream:0 pts:0 pts_time:0 frame:0
[flv @ 0x55d359d7a800] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
[segment @ 0x55d359d75080] segment:'/tmp/cache/test1-20221004114836.mp4' count:0 ended
[AVIOContext @ 0x55d359d78a40] Statistics: 1967357 bytes written, 2 seeks, 11 writeouts
[flv @ 0x55d359d7a800] Failed to update header with correct duration.
[flv @ 0x55d359d7a800] Failed to update header with correct filesize.
frame= 302 fps= 22 q=-1.0 Lq=-1.0 size=N/A time=00:00:14.91 bitrate=N/A speed=1.07x
video:3834kB audio:57kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Input file #0 (rtsp://REDACTED:[email protected]:554/Streaming/Channels/101):
Input stream #0:0 (video): 302 packets read (1962826 bytes);
Input stream #0:1 (audio): 237 packets read (58019 bytes);
Total: 539 packets (2020845 bytes) demuxed
Output file #0 (/tmp/cache/test1-%Y%m%d%H%M%S.mp4):
Output stream #0:0 (video): 302 packets muxed (1962826 bytes);
Total: 302 packets (1962826 bytes) muxed
Output file #1 (rtmp://127.0.0.1/live/test1):
Output stream #1:0 (video): 302 packets muxed (1962826 bytes);
Output stream #1:1 (audio): 237 packets muxed (58019 bytes);
Total: 539 packets (2020845 bytes) muxed
[AVIOContext @ 0x55d359dac5c0] Statistics: 2031358 bytes written, 0 seeks, 557 writeouts
Exiting normally, received signal 2.
The exit was because I pressed CTRL+C after a short while.
As far as I can see this is only using software decode rather than hardware decode as displayed here:
Stream #0:0: Video: h264 (Main), 1 reference frame, yuv420p(progressive, left), 1920x1080 (1920x1088), 20 fps, 20 tbr, 90k tbn
Any help would be fantastic as I have looked at so many options both using QSV and VAAPI but neither seem to work within the addon and I am wondering if this is even possible within the addon ?
Render device:
root@ccab4aaf-frigate:/opt/frigate# ls -l /dev/dri/
total 0
drwxr-xr-x 2 root root 80 Oct 4 10:02 by-path
crw-rw---- 1 root 28 226, 0 Oct 4 10:02 card0
crw-rw-rw- 1 root 103 226, 128 Oct 4 10:02 renderD128
If there is more that would help please let me know I would love to get this working.
Lee
What CPU do you have?
Also those QSV args are deprecated, new args are suggested https://docs.frigate.video/configuration/hardware_acceleration#intel-based-cpus-10th-generation-via-quicksync-1
Hi Nick,
Thanks for coming back, I did include the CPU model in the top section but its easily missed. The CPU is: 11th Gen Intel(R) Core(TM) i5-11400T.
Okay cool, definitely try the args I linked instead.
Hi Nick,
I ran the updated command with the hardware options recommended within the frigate container:
ffmpeg -hide_banner -loglevel verbose -c:v h264_qsv -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsp://REDACTED:[email protected]:554/Streaming/Channels/101 -f segment -segment_time 60 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -an /tmp/cache/test1-%Y%m%d%H%M%S.mp4 -c copy -f flv rtmp://127.0.0.1/live/test1
Output below:
root@ccab4aaf-frigate:/opt/frigate# ffmpeg -hide_banner -loglevel verbose -c:v h264_qsv -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsp://REDACTED:[email protected]:554/Streaming/Channels/101 -f segment -segment_time 60 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -an /tmp/cache/test1-%Y%m%d%H%M%S.mp4 -c copy -f flv rtmp://127.0.0.1/live/test1
[tcp @ 0x55be2e70b180] Starting connection attempt to 192.168.1.84 port 554
[tcp @ 0x55be2e70b180] Successfully connected to 192.168.1.84 port 554
[rtsp @ 0x55be2e707ec0] SDP:
v=0
o=- 1664889138402813 1664889138402813 IN IP4 192.168.1.84
s=Media Presentation
e=NONE
b=AS:5100
t=0 0
a=control:rtsp://192.168.1.84:554/Streaming/Channels/101/
m=video 0 RTP/AVP 96
b=AS:5000
a=control:rtsp://192.168.1.84:554/Streaming/Channels/101/trackID=1
a=rtpmap:96 H264/90000
a=fmtp:96 profile-level-id=420029; packetization-mode=1; sprop-parameter-sets=Z00AKY2NQDwBE/LCAAARlAACvyAI,aO44gA==
m=audio 0 RTP/AVP 104
b=AS:50
a=control:rtsp://192.168.1.84:554/Streaming/Channels/101/trackID=2
a=rtpmap:104 mpeg4-generic/16000/1
a=fmtp:104 profile-level-id=15; streamtype=5; mode=AAC-hbr; config=1408;SizeLength=13; IndexLength=3; IndexDeltaLength=3; Profile=1;
a=Media_header:MEDIAINFO=494D4B48010100000400010001200110803E0000007D000000000000000000000000000000000000;
a=appversion:1.0
[rtsp @ 0x55be2e707ec0] setting jitter buffer size to 0
Last message repeated 1 times
[h264 @ 0x55be2e70d300] Reinit context to 1920x1088, pix_fmt: yuv420p
Input #0, rtsp, from 'rtsp://REDACTED:[email protected]:554/Streaming/Channels/101':
Metadata:
title : Media Presentation
Duration: N/A, start: 1664889139.787375, bitrate: N/A
Stream #0:0: Video: h264 (Main), 1 reference frame, yuv420p(progressive, left), 1920x1080 (1920x1088), 20 fps, 20 tbr, 90k tbn
Stream #0:1: Audio: aac (LC), 16000 Hz, mono, fltp
[tcp @ 0x55be2e716380] Starting connection attempt to 127.0.0.1 port 1935
[tcp @ 0x55be2e716380] Successfully connected to 127.0.0.1 port 1935
[segment @ 0x55be2e725e80] Selected stream id:0 type:video
[segment @ 0x55be2e725e80] Opening '/tmp/cache/test1-20221004141220.mp4' for writing
Output #0, segment, to '/tmp/cache/test1-%Y%m%d%H%M%S.mp4':
Metadata:
title : Media Presentation
encoder : Lavf59.27.100
Stream #0:0: Video: h264 (Main), 1 reference frame, yuv420p(progressive, left), 1920x1080 (0x0), q=2-31, 20 fps, 20 tbr, 10240 tbn
Output #1, flv, to 'rtmp://127.0.0.1/live/test1':
Metadata:
title : Media Presentation
encoder : Lavf59.27.100
Stream #1:0: Video: h264 (Main), 1 reference frame ([7][0][0][0] / 0x0007), yuv420p(progressive, left), 1920x1080 (0x0), q=2-31, 20 fps, 20 tbr, 1k tbn
Stream #1:1: Audio: aac (LC) ([10][0][0][0] / 0x000A), 16000 Hz, mono, fltp
Stream mapping:
Stream #0:0 -> #0:0 (copy)
Stream #0:0 -> #1:0 (copy)
Stream #0:1 -> #1:1 (copy)
Press [q] to stop, [?] for help
[segment @ 0x55be2e725e80] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
[segment @ 0x55be2e725e80] segment:'/tmp/cache/test1-20221004141220.mp4' starts with packet stream:0 pts:0 pts_time:0 frame:0
[flv @ 0x55be2e723980] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
[segment @ 0x55be2e725e80] Non-monotonous DTS in output stream 0:0; previous: 3034, current: 3034; changing to 3035. This may result in incorrect timestamps in the output file.
[segment @ 0x55be2e725e80] Non-monotonous DTS in output stream 0:0; previous: 11020, current: 11020; changing to 11021. This may result in incorrect timestamps in the output file.
[segment @ 0x55be2e725e80] segment:'/tmp/cache/test1-20221004141220.mp4' count:0 ended
[AVIOContext @ 0x55be2e75a900] Statistics: 1343300 bytes written, 2 seeks, 9 writeouts
[flv @ 0x55be2e723980] Failed to update header with correct duration.
[flv @ 0x55be2e723980] Failed to update header with correct filesize.
frame= 202 fps= 23 q=-1.0 Lq=-1.0 size=N/A time=00:00:10.03 bitrate=N/A speed=1.13x
video:2617kB audio:38kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Input file #0 (rtsp://REDACTED:[email protected]:554/Streaming/Channels/101):
Input stream #0:0 (video): 202 packets read (1339989 bytes);
Input stream #0:1 (audio): 159 packets read (39036 bytes);
Total: 361 packets (1379025 bytes) demuxed
Output file #0 (/tmp/cache/test1-%Y%m%d%H%M%S.mp4):
Output stream #0:0 (video): 202 packets muxed (1339989 bytes);
Total: 202 packets (1339989 bytes) muxed
Output file #1 (rtmp://127.0.0.1/live/test1):
Output stream #1:0 (video): 202 packets muxed (1339989 bytes);
Output stream #1:1 (audio): 159 packets muxed (39036 bytes);
Total: 361 packets (1379025 bytes) muxed
[AVIOContext @ 0x55be2e7295c0] Statistics: 1386212 bytes written, 0 seeks, 371 writeouts
Exiting normally, received signal 2.
The Frigate log just shows:
[2022-10-04 14:21:41] frigate.app INFO : Starting Frigate (0.11.0-3846a13)
[2022-10-04 14:21:41] frigate.app INFO : Creating directory: /tmp/cache
Starting migrations
[2022-10-04 14:21:41] peewee_migrate INFO : Starting migrations
There is nothing to migrate
[2022-10-04 14:21:41] peewee_migrate INFO : There is nothing to migrate
[2022-10-04 14:21:41] detector.coral INFO : Starting detection process: 232
[2022-10-04 14:21:41] frigate.app INFO : Output process started: 234
[2022-10-04 14:21:41] frigate.edgetpu INFO : Attempting to load TPU as usb
[2022-10-04 14:21:41] ws4py INFO : Using epoll
[2022-10-04 14:21:41] frigate.app INFO : Camera processor started for living_room: 239
[2022-10-04 14:21:41] frigate.app INFO : Camera processor started for study: 242
[2022-10-04 14:21:41] frigate.app INFO : Camera processor started for kids_room: 244
[2022-10-04 14:21:41] frigate.app INFO : Camera processor started for front_door: 246
[2022-10-04 14:21:41] frigate.app INFO : Camera processor started for back_garden: 248
[2022-10-04 14:21:41] frigate.app INFO : Camera processor started for kitchen: 249
[2022-10-04 14:21:41] frigate.app INFO : Camera processor started for parking: 251
[2022-10-04 14:21:41] frigate.app INFO : Capture process started for living_room: 252
[2022-10-04 14:21:41] frigate.app INFO : Capture process started for study: 255
[2022-10-04 14:21:41] frigate.app INFO : Capture process started for kids_room: 258
[2022-10-04 14:21:41] frigate.app INFO : Capture process started for front_door: 263
[2022-10-04 14:21:41] frigate.app INFO : Capture process started for back_garden: 273
[2022-10-04 14:21:41] frigate.app INFO : Capture process started for kitchen: 285
[2022-10-04 14:21:41] frigate.app INFO : Capture process started for parking: 292
[2022-10-04 14:21:42] ws4py INFO : Using epoll
[2022-10-04 14:21:44] frigate.edgetpu INFO : TPU found
[2022-10-04 14:22:01] watchdog.living_room INFO : No frames received from living_room in 20 seconds. Exiting ffmpeg...
[2022-10-04 14:22:01] watchdog.living_room INFO : Waiting for ffmpeg to exit gracefully...
[2022-10-04 14:22:01] watchdog.study INFO : No frames received from study in 20 seconds. Exiting ffmpeg...
[2022-10-04 14:22:01] watchdog.study INFO : Waiting for ffmpeg to exit gracefully...
[2022-10-04 14:22:01] watchdog.kids_room INFO : No frames received from kids_room in 20 seconds. Exiting ffmpeg...
[2022-10-04 14:22:01] watchdog.kids_room INFO : Waiting for ffmpeg to exit gracefully...
[2022-10-04 14:22:01] watchdog.front_door INFO : No frames received from front_door in 20 seconds. Exiting ffmpeg...
[2022-10-04 14:22:01] watchdog.front_door INFO : Waiting for ffmpeg to exit gracefully...
[2022-10-04 14:22:01] watchdog.back_garden INFO : No frames received from back_garden in 20 seconds. Exiting ffmpeg...
[2022-10-04 14:22:01] watchdog.back_garden INFO : Waiting for ffmpeg to exit gracefully...
[2022-10-04 14:22:01] watchdog.kitchen INFO : No frames received from kitchen in 20 seconds. Exiting ffmpeg...
Try running vainfo
in the container and paste the output here
I had tried that but vainfo does not exist in the container.
# docker exec -it 495914c8e5a6 bash -l
root@ccab4aaf-frigate:/opt/frigate# vainfo
bash: vainfo: command not found
root@ccab4aaf-frigate:/opt/frigate# find / -iname vainfo
okay then run apt-get update
apt-get install vainfo -y
vainfo
I just tried installing it into the container and the output is below:
root@ccab4aaf-frigate:/opt/frigate# apt-get update
Get:1 http://deb.debian.org/debian bullseye InRelease [116 kB]
Get:2 http://raspbian.raspberrypi.org/raspbian bullseye InRelease [15.0 kB]
Get:3 http://deb.debian.org/debian-security bullseye-security InRelease [48.4 kB]
Get:4 http://deb.debian.org/debian bullseye-updates InRelease [44.1 kB]
Get:5 http://deb.debian.org/debian bullseye/contrib amd64 Packages [50.6 kB]
Get:6 http://deb.debian.org/debian bullseye/non-free amd64 Packages [97.7 kB]
Get:7 http://deb.debian.org/debian bullseye/main amd64 Packages [8184 kB]
Get:8 https://packages.cloud.google.com/apt coral-edgetpu-stable InRelease [6722 B]
Ign:9 https://packages.cloud.google.com/apt coral-edgetpu-stable/main amd64 Packages
Get:10 http://deb.debian.org/debian-security bullseye-security/main amd64 Packages [186 kB]
Get:11 http://deb.debian.org/debian-security bullseye-security/non-free amd64 Packages [528 B]
Get:12 http://deb.debian.org/debian bullseye-updates/main amd64 Packages [6344 B]
Get:9 https://packages.cloud.google.com/apt coral-edgetpu-stable/main amd64 Packages [2317 B]
Fetched 8758 kB in 2s (4728 kB/s)
Reading package lists... Done
N: Skipping acquire of configured file 'main/binary-amd64/Packages' as repository 'http://raspbian.raspberrypi.org/raspbian bullseye InRelease' doesn't support architecture 'amd64'
N: Skipping acquire of configured file 'non-free/binary-amd64/Packages' as repository 'http://raspbian.raspberrypi.org/raspbian bullseye InRelease' doesn't support architecture 'amd64'
N: Skipping acquire of configured file 'contrib/binary-amd64/Packages' as repository 'http://raspbian.raspberrypi.org/raspbian bullseye InRelease' doesn't support architecture 'amd64'
N: Skipping acquire of configured file 'rpi/binary-amd64/Packages' as repository 'http://raspbian.raspberrypi.org/raspbian bullseye InRelease' doesn't support architecture 'amd64'
root@ccab4aaf-frigate:/opt/frigate# apt search vainfo
Sorting... Done
Full Text Search... Done
vainfo/stable 2.10.0+ds1-1 amd64
Video Acceleration (VA) API for Linux -- info program```
Installed into container
```root@ccab4aaf-frigate:/opt/frigate# apt install vainfo
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
libva-wayland2 libva-x11-2 libwayland-client0 libxext6 libxfixes3
The following NEW packages will be installed:
libva-wayland2 libva-x11-2 libwayland-client0 libxext6 libxfixes3 vainfo
0 upgraded, 6 newly installed, 0 to remove and 2 not upgraded.
Need to get 166 kB of archives.
After this operation, 446 kB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://deb.debian.org/debian bullseye/main amd64 libwayland-client0 amd64 1.18.0-2~exp1.1 [26.9 kB]
Get:2 http://deb.debian.org/debian bullseye/main amd64 libva-wayland2 amd64 2.10.0-1 [20.9 kB]
Get:3 http://deb.debian.org/debian bullseye/main amd64 libxext6 amd64 2:1.3.3-1.1 [52.7 kB]
Get:4 http://deb.debian.org/debian bullseye/main amd64 libxfixes3 amd64 1:5.0.3-2 [22.1 kB]
Get:5 http://deb.debian.org/debian bullseye/main amd64 libva-x11-2 amd64 2.10.0-1 [24.0 kB]
Get:6 http://deb.debian.org/debian bullseye/main amd64 vainfo amd64 2.10.0+ds1-1 [19.0 kB]
Fetched 166 kB in 0s (1786 kB/s)
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package libwayland-client0:amd64.
(Reading database ... 11766 files and directories currently installed.)
Preparing to unpack .../0-libwayland-client0_1.18.0-2~exp1.1_amd64.deb ...
Unpacking libwayland-client0:amd64 (1.18.0-2~exp1.1) ...
Selecting previously unselected package libva-wayland2:amd64.
Preparing to unpack .../1-libva-wayland2_2.10.0-1_amd64.deb ...
Unpacking libva-wayland2:amd64 (2.10.0-1) ...
Selecting previously unselected package libxext6:amd64.
Preparing to unpack .../2-libxext6_2%3a1.3.3-1.1_amd64.deb ...
Unpacking libxext6:amd64 (2:1.3.3-1.1) ...
Selecting previously unselected package libxfixes3:amd64.
Preparing to unpack .../3-libxfixes3_1%3a5.0.3-2_amd64.deb ...
Unpacking libxfixes3:amd64 (1:5.0.3-2) ...
Selecting previously unselected package libva-x11-2:amd64.
Preparing to unpack .../4-libva-x11-2_2.10.0-1_amd64.deb ...
Unpacking libva-x11-2:amd64 (2.10.0-1) ...
Selecting previously unselected package vainfo.
Preparing to unpack .../5-vainfo_2.10.0+ds1-1_amd64.deb ...
Unpacking vainfo (2.10.0+ds1-1) ...
Setting up libxext6:amd64 (2:1.3.3-1.1) ...
Setting up libxfixes3:amd64 (1:5.0.3-2) ...
Setting up libva-x11-2:amd64 (2.10.0-1) ...
Setting up libwayland-client0:amd64 (1.18.0-2~exp1.1) ...
Setting up libva-wayland2:amd64 (2.10.0-1) ...
Setting up vainfo (2.10.0+ds1-1) ...
Processing triggers for libc-bin (2.31-13+deb11u4) ...```
Output from vainfo:
```root@ccab4aaf-frigate:/opt/frigate# vainfo
error: XDG_RUNTIME_DIR not set in the environment.
error: can't connect to X server!
libva info: VA-API version 1.10.0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_10
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.10 (libva 2.10.0)
vainfo: Driver version: Intel iHD driver for Intel(R) Gen Graphics - 21.1.1 ()
vainfo: Supported profile and entrypoints
VAProfileNone : VAEntrypointVideoProc
VAProfileNone : VAEntrypointStats
VAProfileMPEG2Simple : VAEntrypointVLD
VAProfileMPEG2Simple : VAEntrypointEncSlice
VAProfileMPEG2Main : VAEntrypointVLD
VAProfileMPEG2Main : VAEntrypointEncSlice
VAProfileH264Main : VAEntrypointVLD
VAProfileH264Main : VAEntrypointEncSlice
VAProfileH264Main : VAEntrypointFEI
VAProfileH264Main : VAEntrypointEncSliceLP
VAProfileH264High : VAEntrypointVLD
VAProfileH264High : VAEntrypointEncSlice
VAProfileH264High : VAEntrypointFEI
VAProfileH264High : VAEntrypointEncSliceLP
VAProfileVC1Simple : VAEntrypointVLD
VAProfileVC1Main : VAEntrypointVLD
VAProfileVC1Advanced : VAEntrypointVLD
VAProfileJPEGBaseline : VAEntrypointVLD
VAProfileJPEGBaseline : VAEntrypointEncPicture
VAProfileH264ConstrainedBaseline: VAEntrypointVLD
VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice
VAProfileH264ConstrainedBaseline: VAEntrypointFEI
VAProfileH264ConstrainedBaseline: VAEntrypointEncSliceLP
VAProfileHEVCMain : VAEntrypointVLD
VAProfileHEVCMain : VAEntrypointEncSlice
VAProfileHEVCMain : VAEntrypointFEI
VAProfileHEVCMain : VAEntrypointEncSliceLP
VAProfileHEVCMain10 : VAEntrypointVLD
VAProfileHEVCMain10 : VAEntrypointEncSlice
VAProfileHEVCMain10 : VAEntrypointEncSliceLP
VAProfileVP9Profile0 : VAEntrypointVLD
VAProfileVP9Profile1 : VAEntrypointVLD
VAProfileVP9Profile2 : VAEntrypointVLD
VAProfileVP9Profile3 : VAEntrypointVLD
VAProfileHEVCMain12 : VAEntrypointVLD
VAProfileHEVCMain12 : VAEntrypointEncSlice
VAProfileHEVCMain422_10 : VAEntrypointVLD
VAProfileHEVCMain422_10 : VAEntrypointEncSlice
VAProfileHEVCMain422_12 : VAEntrypointVLD
VAProfileHEVCMain422_12 : VAEntrypointEncSlice
VAProfileHEVCMain444 : VAEntrypointVLD
VAProfileHEVCMain444 : VAEntrypointEncSliceLP
VAProfileHEVCMain444_10 : VAEntrypointVLD
VAProfileHEVCMain444_10 : VAEntrypointEncSliceLP
VAProfileHEVCMain444_12 : VAEntrypointVLD
VAProfileHEVCSccMain : VAEntrypointVLD
VAProfileHEVCSccMain : VAEntrypointEncSliceLP
VAProfileHEVCSccMain10 : VAEntrypointVLD
VAProfileHEVCSccMain10 : VAEntrypointEncSliceLP
VAProfileHEVCSccMain444 : VAEntrypointVLD
VAProfileHEVCSccMain444 : VAEntrypointEncSliceLP
VAProfileAV1Profile0 : VAEntrypointVLD
VAProfileHEVCSccMain444_10 : VAEntrypointVLD
VAProfileHEVCSccMain444_10 : VAEntrypointEncSliceLP
Interesting, looks like you're probably running into https://github.com/blakeblackshear/frigate/issues/3941
If you manually update the driver it might get it working
How would I go about updating the driver ?
See https://github.com/blakeblackshear/frigate/issues/3941#issuecomment-1259531976
MMmm I updated the driver but have not figured out how to restart the frigate to test as restarting frigate removes the new driver and the vainfo tool.
Strangely I can't get the intel-gpu-tool to run within docker but this may not be surprising I am guessing elevated privileges are required.
Do you know how I can restart frigate without restarting the docker image ?
Hm, at least with standard docker the changes remain when restarting they just go away when pulling a new image. Perhaps HA OS is removing the image on each restart.
If you apt-get install intel-gpu-tool
then run `intel_gpu_top it should work. What error are you getting when you try?
I suspect it is doing that which makes it more tricky.
Below is what I get for intel_gpu_top
root@ccab4aaf-frigate:/opt/frigate# intel_gpu_top
Failed to initialize PMU! (Permission denied)
Are you running full access? And do you have protected mode disabled?
Mmm no not running full access will try that.
Thats with full access and protection mode off.
root@ccab4aaf-frigate-fa:/opt/frigate# intel_gpu_top
Failed to initialize PMU! (Permission denied)
Running a system trace shows:
root@ccab4aaf-frigate-fa:/opt/frigate# strace intel_gpu_top
execve("/usr/bin/intel_gpu_top", ["intel_gpu_top"], 0x7ffe6c8f0d70 /* 13 vars */) = 0
brk(NULL) = 0x5592a7a31000
access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=14342, ...}) = 0
mmap(NULL, 14342, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f3be99c3000
close(3) = 0
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0@>\2\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=1905632, ...}) = 0
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f3be99c1000
mmap(NULL, 1918592, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f3be97ec000
mmap(0x7f3be980e000, 1417216, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x22000) = 0x7f3be980e000
mmap(0x7f3be9968000, 323584, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x17c000) = 0x7f3be9968000
mmap(0x7f3be99b7000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1ca000) = 0x7f3be99b7000
mmap(0x7f3be99bd000, 13952, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f3be99bd000
close(3) = 0
arch_prctl(ARCH_SET_FS, 0x7f3be99c2580) = 0
mprotect(0x7f3be99b7000, 16384, PROT_READ) = 0
mprotect(0x5592a676f000, 4096, PROT_READ) = 0
mprotect(0x7f3be99f1000, 4096, PROT_READ) = 0
munmap(0x7f3be99c3000, 14342) = 0
ioctl(1, TCGETS, {B38400 opost isig icanon echo ...}) = 0
brk(NULL) = 0x5592a7a31000
brk(0x5592a7a52000) = 0x5592a7a52000
openat(AT_FDCWD, "/sys/devices/i915/events", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 3
fstat(3, {st_mode=S_IFDIR|0755, st_size=0, ...}) = 0
getdents64(3, 0x5592a7a31500 /* 35 entries */, 32768) = 1296
openat(3, "rcs0-busy", O_RDONLY) = 4
read(4, "config=0x0\n", 128) = 11
close(4) = 0
openat(3, "vecs0-busy", O_RDONLY) = 4
read(4, "config=0x3000\n", 128) = 14
close(4) = 0
openat(3, "vcs0-busy", O_RDONLY) = 4
read(4, "config=0x2000\n", 128) = 14
close(4) = 0
openat(3, "bcs0-busy", O_RDONLY) = 4
read(4, "config=0x1000\n", 128) = 14
close(4) = 0
getdents64(3, 0x5592a7a31500 /* 0 entries */, 32768) = 0
openat(AT_FDCWD, "/sys/bus/event_source/devices/i915/type", O_RDONLY) = 4
read(4, "22\n", 63) = 3
close(4) = 0
openat(AT_FDCWD, "/sys/devices/system/cpu", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 4
fstat(4, {st_mode=S_IFDIR|0755, st_size=0, ...}) = 0
getdents64(4, 0x5592a7a3a630 /* 30 entries */, 32768) = 864
getdents64(4, 0x5592a7a3a630 /* 0 entries */, 32768) = 0
close(4) = 0
perf_event_open({type=0x16 /* PERF_TYPE_??? */, size=PERF_ATTR_SIZE_VER6, config=0x100002, ...}, -1, 0, -1, 0) = -1 EACCES (Permission denied)
write(2, "Failed to initialize PMU! (Permi"..., 46Failed to initialize PMU! (Permission denied)
) = 46
exit_group(1) = ?
+++ exited with 1 +++
Okay thanks for trying, that is unfortunate. To confirm, hwaccel is still not working with the full access and no protected mode?
Yep that's confirmed:
- Full Access
- Protected mode is off
With the hwaccel option: hwaccel_args: -c:v h264_qsv
I get:
[2022-10-04 23:45:13] ffmpeg.kitchen.detect ERROR : Error while decoding stream #0:0: Input/output error
[2022-10-04 23:45:13] ffmpeg.kitchen.detect ERROR : cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[2022-10-04 23:45:13] ffmpeg.kitchen.detect ERROR : Last message repeated 2 times
[2022-10-04 23:45:13] ffmpeg.kitchen.detect ERROR : [h264_qsv @ 0x55d44194d440] Error during QSV decoding.: device failed (-17)
[2022-10-04 23:45:13] ffmpeg.kitchen.detect ERROR : Error while decoding stream #0:0: Input/output error
[2022-10-04 23:45:13] ffmpeg.kitchen.detect ERROR : cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[2022-10-04 23:45:13] ffmpeg.kitchen.detect ERROR : Last message repeated 2 times
Plus a number of other messages
with hwaccel args:
hwaccel_args: - -hwaccel - vaapi - -hwaccel_output_format - vaapi - -vaapi_device - /dev/dri/renderD128
I get:
[2022-10-04 23:49:28] ffmpeg.kitchen.detect ERROR : [h264 @ 0x55e207c707c0] hardware accelerator failed to decode picture
[2022-10-04 23:49:28] ffmpeg.kitchen.detect ERROR : cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[2022-10-04 23:49:28] ffmpeg.kitchen.detect ERROR : Error while decoding stream #0:0: Input/output error
[2022-10-04 23:49:28] ffmpeg.kitchen.detect ERROR : [h264 @ 0x55e207be8a40] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 2
[2022-10-04 23:49:28] ffmpeg.kitchen.detect ERROR : cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[2022-10-04 23:49:28] ffmpeg.kitchen.detect ERROR : [h264 @ 0x55e207be8a40] Param buffer (type 0, 672 bytes) is 0.
[2022-10-04 23:49:28] ffmpeg.kitchen.detect ERROR : [h264 @ 0x55e207be8a40] Param buffer (type 1, 240 bytes) is 0x1.
[2022-10-04 23:49:28] ffmpeg.kitchen.detect ERROR : cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[2022-10-04 23:49:28] ffmpeg.kitchen.detect ERROR : [h264 @ 0x55e207be8a40] Slice 0 param buffer (3128 bytes) is 0x2.
[2022-10-04 23:49:28] ffmpeg.kitchen.detect ERROR : [h264 @ 0x55e207be8a40] Slice 0 data buffer (33 bytes) is 0x3.
[2022-10-04 23:49:28] ffmpeg.kitchen.detect ERROR : [h264 @ 0x55e207be8a40] Decode to surface 0x16.
The above errors repeat in both cases.