frigate
frigate copied to clipboard
[Support]: 1 out of 4 Edge TPUs not detected
Describe the problem you are having
I have 2 Dual Edge TPUs installed in MBS's PCIE adapters. I'm running Frigate as a Hassio addon for Home Assistant.
Frigate consistently reports for pci:2 "No EdgeTPU was detected. If you do not have a Coral device yet, you must configure CPU detectors."
This occurs at pci:2 even when the 2 dual TPUs & adapters are swapped, so I'm not convinced it's a hardware issue.
Linux shows all 4 apex devices:
dmesg | grep apex
[ 11.051976] apex 0000:04:00.0: Apex performance not throttled due to temperature
[ 11.051984] apex 0000:03:00.0: Apex performance not throttled due to temperature
[ 11.306691] apex 0000:08:00.0: Apex performance not throttled due to temperature
[ 11.310702] apex 0000:09:00.0: Apex performance not throttled due to temperature
ls /sys/class/apex/
apex_0 apex_1 apex_2 apex_3
I suspect it might be a permissions issue, but the permissions on the devices are all identical:
ls -l /dev/apex*
crw-rw---- 1 root apex 120, 0 Jul 26 16:17 /dev/apex_0
crw-rw---- 1 root apex 120, 1 Jul 26 16:17 /dev/apex_1
crw-rw---- 1 root apex 120, 2 Jul 26 16:17 /dev/apex_2
crw-rw---- 1 root apex 120, 3 Jul 26 16:17 /dev/apex_3
Version
0.10.1
Frigate config file
detectors:
coral1:
type: edgetpu
device: pci:0
coral2:
type: edgetpu
device: pci:1
coral3:
type: edgetpu
device: pci:2
coral4:
type: edgetpu
device: pci:3
mqtt:
# Required: host name
host: homeautomation.lan
# Optional: port (default: shown below)
port: 1883
# Optional: topic prefix (default: shown below)
# WARNING: must be unique if you are running multiple instances
topic_prefix: frigate
# Optional: client id (default: shown below)
# WARNING: must be unique if you are running multiple instances
client_id: frigate
# Optional: user
user: frigate
# Optional: password
# NOTE: Environment variables that begin with 'FRIGATE_' may be referenced in {}.
# eg. password: '{FRIGATE_MQTT_PASSWORD}'
password: xxxxx
# Optional: interval in seconds for publishing stats (default: shown below)
stats_interval: 60
record:
enabled: True
retain:
days: 14
events:
retain:
default: 20
cameras:
# Name of your camera
rear_hall:
ffmpeg:
hwaccel_args: -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p
output_args:
detect: -vf transpose=1 -f rawvideo -pix_fmt yuv420p
inputs:
- path: rtsp://10.0.2.12:554/user=admin_password=xxxxx_channel=1_stream=0.sdp?real_stream
roles:
- detect
- rtmp
- record
motion:
mask:
- 1200,452,1201,636,1164,637,1157,449
zones:
stairs:
coordinates: 0,124,168,312,491,304,508,424,463,590,0,720
hall:
coordinates: 1280,720,1280,271,834,295,499,300,516,462,472,720
objects:
track:
- person
garage_west:
ffmpeg:
hwaccel_args: -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p
inputs:
- path: rtsp://10.0.2.10/user=admin_password=xxxxx_channel=1_stream=0.sdp?real_stream
roles:
- detect
- rtmp
- record
zones:
parking:
coordinates: 0,223,291,153,748,84,1049,53,1028,21,727,34,542,58,216,110,0,157
footpath:
coordinates: 459,124,1005,62,1056,76,1280,76,1280,148,973,217,479,339,0,464,0,224
driveway:
coordinates: 1280,720,1280,295,977,225,0,471,0,720
objects:
track:
- person
- car
- bicycle
- boat
- cat
- dog
- backpack
- handbag
- suitcase
- bottle
- cell phone
garage_east:
ffmpeg:
hwaccel_args: -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p
inputs:
- path: rtsp://10.0.2.11/user=admin_password=xxxxx_channel=1_stream=0.sdp?real_stream
roles:
- detect
- rtmp
- record
zones:
parking:
coordinates: 0,223,291,153,748,84,1049,53,1028,21,727,34,542,58,216,110,0,157
driveway:
coordinates: 0,226,0,720,1280,720,1280,307,861,191,669,229,554,232,443,225,366,227,194,174
objects:
track:
- person
- car
- bicycle
- boat
- cat
- dog
- backpack
- handbag
- suitcase
- bottle
- cell phone
motion:
mask:
- 1280,0,1280,77,1054,74,1039,23,982,33,939,22,768,28,604,48,400,80,204,115,0,159,0,0
study:
ffmpeg:
hwaccel_args: -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p
inputs:
- path: rtsp://10.0.2.14:554/user=admin_password=xxxxxx_channel=1_stream=0.sdp?real_stream
roles:
- detect
- rtmp
- record
zones:
study_all:
coordinates: 0,0,1280,0,1280,720,0,720
objects:
track:
- person
- backpack
- handbag
- suitcase
- bottle
- cell phone
motion:
mask:
- 966,73,1280,71,1280,42,957,44
workshop_east_cam:
ffmpeg:
hwaccel_args: -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p
inputs:
- path: rtsp://10.0.2.15:554/user=admin_password=xxxxxx_channel=1_stream=0.sdp?real_stream
roles:
- detect
- rtmp
- record
zones:
workshop_west:
coordinates: 0,315,1280,304,1280,0,0,0
workshop_east:
coordinates: 0,331,1172,285,1174,274,1280,158,1280,720,0,720
objects:
track:
- person
- backpack
- handbag
- suitcase
- bottle
- cell phone
motion:
mask:
- 808,70,1126,67,1127,38,807,36
Relevant log output
[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] done.
[services.d] starting services
[services.d] done.
[2022-07-26 16:18:59] frigate.app INFO : Starting Frigate (0.10.1-83481af)
[2022-07-26 16:18:59] frigate.app INFO : Creating directory: /tmp/cache
Starting migrations
[2022-07-26 16:18:59] peewee_migrate INFO : Starting migrations
There is nothing to migrate
[2022-07-26 16:19:00] peewee_migrate INFO : There is nothing to migrate
[2022-07-26 16:19:00] detector.coral1 INFO : Starting detection process: 217
[2022-07-26 16:19:00] frigate.edgetpu INFO : Attempting to load TPU as pci:0
[2022-07-26 16:19:00] detector.coral2 INFO : Starting detection process: 218
[2022-07-26 16:19:00] frigate.edgetpu INFO : Attempting to load TPU as pci:1
Process detector:coral3:
[2022-07-26 16:19:00] frigate.edgetpu INFO : TPU found
[2022-07-26 16:19:00] detector.coral4 INFO : Starting detection process: 224
[2022-07-26 16:19:00] frigate.edgetpu INFO : Attempting to load TPU as pci:3
[2022-07-26 16:19:00] frigate.edgetpu INFO : TPU found
[2022-07-26 16:19:00] frigate.app INFO : Output process started: 226
[2022-07-26 16:19:00] frigate.app INFO : Camera processor started for rear_hall: 243
[2022-07-26 16:19:00] frigate.app INFO : Camera processor started for garage_west: 245
[2022-07-26 16:19:00] frigate.app INFO : Camera processor started for garage_east: 246
[2022-07-26 16:19:00] frigate.app INFO : Camera processor started for study: 248
[2022-07-26 16:19:00] frigate.app INFO : Camera processor started for workshop_east_cam: 259
[2022-07-26 16:19:00] frigate.app INFO : Capture process started for rear_hall: 260
[2022-07-26 16:19:00] frigate.app INFO : Capture process started for garage_west: 261
[2022-07-26 16:19:00] frigate.app INFO : Capture process started for garage_east: 262
[2022-07-26 16:19:00] frigate.app INFO : Capture process started for study: 264
[2022-07-26 16:19:00] frigate.app INFO : Capture process started for workshop_east_cam: 265
[2022-07-26 16:19:00] ws4py INFO : Using epoll
[2022-07-26 16:19:00] detector.coral3 INFO : Starting detection process: 222
[2022-07-26 16:19:00] frigate.edgetpu INFO : Attempting to load TPU as pci:2
[2022-07-26 16:19:00] frigate.edgetpu ERROR : No EdgeTPU was detected. If you do not have a Coral device yet, you must configure CPU detectors.
[2022-07-26 16:19:00] frigate.edgetpu INFO : TPU found
[2022-07-26 16:19:00] ws4py INFO : Using epoll
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 160, in load_delegate
delegate = Delegate(library, options)
File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 119, in __init__
raise ValueError(capture.message)
ValueError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/opt/frigate/frigate/edgetpu.py", line 136, in run_detector
object_detector = LocalObjectDetector(
File "/opt/frigate/frigate/edgetpu.py", line 44, in __init__
edge_tpu_delegate = load_delegate("libedgetpu.so.1.0", device_config)
File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 162, in load_delegate
raise ValueError('Failed to load delegate from {}\n{}'.format(
ValueError: Failed to load delegate from libedgetpu.so.1.0
[2022-07-26 16:19:20] frigate.watchdog INFO : Detection appears to have stopped. Exiting frigate...
[2022-07-26 16:19:20] frigate.app INFO : Stopping...
[2022-07-26 16:19:20] ws4py INFO : Closing all websockets with [1001] 'Server is shutting down'
[2022-07-26 16:19:20] frigate.events INFO : Exiting event cleanup...
[2022-07-26 16:19:20] frigate.record INFO : Exiting recording cleanup...
[2022-07-26 16:19:20] frigate.stats INFO : Exiting watchdog...
[2022-07-26 16:19:20] frigate.watchdog INFO : Exiting watchdog...
[2022-07-26 16:19:20] frigate.object_processing INFO : Exiting object processor...
[2022-07-26 16:19:38] frigate.events INFO : Exiting event processor...
[2022-07-26 16:19:38] frigate.record INFO : Exiting recording maintenance...
[2022-07-26 16:19:38] peewee.sqliteq INFO : writer received shutdown request, exiting.
[2022-07-26 16:19:38] root INFO : Waiting for detection process to exit gracefully...
[2022-07-26 16:19:42] root INFO : Waiting for detection process to exit gracefully...
[2022-07-26 16:19:42] root INFO : Waiting for detection process to exit gracefully...
[2022-07-26 16:19:42] root INFO : Waiting for detection process to exit gracefully...
FFprobe output from your camera
n/a
Frigate stats
n/a
Operating system
Other Linux
Install method
HassOS Addon
Coral version
M.2
Network connection
Wired
Camera make and model
various
Any other information that may be helpful
No response
You need to use the beta addon to support 4 corals, currently stable only supports 3
That might not matter for the full access addon. Is it always the same one that fails? Does it work if you only configure the 3 that work?
Yes, it works if I comment out the Coral3 block (the one that fails)
I'll try tomorrow commenting out the coral4 block and seeing if it still works in that configuration.
Commenting out Coral4 but leaving Coral3 fails:
[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] done.
[services.d] starting services
[services.d] done.
[2022-07-27 12:02:40] frigate.app INFO : Starting Frigate (0.10.1-83481af)
[2022-07-27 12:02:40] frigate.app INFO : Creating directory: /tmp/cache
Starting migrations
[2022-07-27 12:02:40] peewee_migrate INFO : Starting migrations
There is nothing to migrate
[2022-07-27 12:02:40] peewee_migrate INFO : There is nothing to migrate
[2022-07-27 12:02:41] detector.coral1 INFO : Starting detection process: 216
[2022-07-27 12:02:41] frigate.edgetpu INFO : Attempting to load TPU as pci:0
[2022-07-27 12:02:41] detector.coral2 INFO : Starting detection process: 217
[2022-07-27 12:02:41] frigate.edgetpu INFO : Attempting to load TPU as pci:1
[2022-07-27 12:02:41] detector.coral3 INFO : Starting detection process: 221
[2022-07-27 12:02:41] frigate.edgetpu INFO : Attempting to load TPU as pci:2
[2022-07-27 12:02:41] frigate.edgetpu ERROR : No EdgeTPU was detected. If you do not have a Coral device yet, you must configure CPU detectors.
Process detector:coral3:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 160, in load_delegate
delegate = Delegate(library, options)
File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 119, in __init__
raise ValueError(capture.message)
ValueError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/opt/frigate/frigate/edgetpu.py", line 136, in run_detector
object_detector = LocalObjectDetector(
File "/opt/frigate/frigate/edgetpu.py", line 44, in __init__
edge_tpu_delegate = load_delegate("libedgetpu.so.1.0", device_config)
File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 162, in load_delegate
raise ValueError('Failed to load delegate from {}\n{}'.format(
ValueError: Failed to load delegate from libedgetpu.so.1.0
[2022-07-27 12:02:41] frigate.app INFO : Output process started: 223
[2022-07-27 12:02:41] ws4py INFO : Using epoll
[2022-07-27 12:02:41] frigate.app INFO : Camera processor started for rear_hall: 228
[2022-07-27 12:02:41] frigate.app INFO : Camera processor started for garage_west: 231
[2022-07-27 12:02:41] frigate.app INFO : Camera processor started for garage_east: 233
[2022-07-27 12:02:41] frigate.app INFO : Camera processor started for study: 234
[2022-07-27 12:02:41] frigate.app INFO : Camera processor started for workshop_east_cam: 236
Also interesting that it's pci:2
which I imagine maps to apex_2
, figure it would be apex_3 that wasn't found if it was related to https://github.com/blakeblackshear/frigate-hass-addons/pull/72
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.