roflcoopter
Viseron 3.0.0b12 Deepstack face recognition Error
Hi, I am using Viseron 3.0.0b12 And continuous 24/7 is working great as it uses fewer resources. I am using Deepstack for object detection and face recognition. The Object Detection is working great, as I am using person detection to record and save mp4. But for face recognition part is giving the below error.
Because of that, The faces are not getting tracked I have a few additional questions,
- How can I save the recordings in different tiers like 4 days in one folder path and older than that till 10 days in another folder path?
- Are there ways In Timeline or any plans to add a feature, where we can export the mp4 file from the selected Start Time to the selected End Time?
- Is there any way, how we can access thumbnails http://192.168.2.109:8080/files/data/thumbnails/tapocam01/121.jpg, the same way we can access recordings http://192.168.2.109:8080/files/data/recordings/tapocam01/06:51:52.mp4 ?
Viseron Config
## Start by adding some cameras
ffmpeg:
camera:
tapocam01: # This value has to be unique across all cameras
name: Living Room
host: XXXXXXXXXXX
port: 554
path: /stream1
username: XXXXXXXX
password: XXXXXXXXX
recorder:
idle_timeout: 60
lookback: 15
max_recording_time: 600
create_event_clip: true
storage:
tiers:
- path: /data
events:
max_age:
days: 14
continuous:
max_age:
days: 4
## Then add an object detector
deepstack:
host: XXXXXXXXXXX
port: 5001
object_detector:
cameras:
tapocam01:
scan_on_motion_only: false
fps: 1
labels:
- label: person
confidence: 0.4
trigger_recorder: true
face_recognition:
save_unknown_faces: true
min_confidence: 0.5
cameras:
tapocam01:
labels:
- person
labels:
- person
mog2:
motion_detector:
cameras:
tapocam01: # Attach detector to the configured camera_2 above
fps: 1
recorder_keepalive: true
max_recorder_keepalive: 60 # keeps recording liver for seconds after motion detection is over
area: 0.08 # we can go uptop 0.001 for triggerring small changes
detect_shadows: false
# history: 500
threshold: 5
width: 1280
height: 720
trigger_recorder: true
nvr:
tapocam01: # Run NVR for Living Room camera
Error Log
viseron3 | Traceback (most recent call last):
viseron3 | File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
viseron3 | self.run()
viseron3 | File "/usr/lib/python3.10/threading.py", line 953, in run
viseron3 | self._target(*self._args, **self._kwargs)
viseron3 | File "/src/viseron/domains/post_processor/__init__.py", line 131, in post_process
viseron3 | self.process(
viseron3 | File "/src/viseron/domains/face_recognition/__init__.py", line 136, in process
viseron3 | self.face_recognition(
viseron3 | File "/src/viseron/components/deepstack/face_recognition.py", line 88, in face_recognition
viseron3 | detection["box"]["x_min"] + x1,
viseron3 | KeyError: 'box'
viseron3 | [31m[2024-12-16 11:42:04] [ERROR ] [viseron.watchdog.thread_watchdog] - Thread viseron.domains.post_processor is dead, restarting[0m
viseron3 | [31m[2024-12-16 11:42:04] [ERROR ] [root] - Uncaught thread exception in thread viseron.domains.post_processor[0m
viseron3 | Traceback (most recent call last):
viseron3 | File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
viseron3 | self.run()
viseron3 | File "/usr/lib/python3.10/threading.py", line 953, in run
viseron3 | self._target(*self._args, **self._kwargs)
viseron3 | File "/src/viseron/domains/post_processor/__init__.py", line 131, in post_process
viseron3 | self.process(
viseron3 | File "/src/viseron/domains/face_recognition/__init__.py", line 136, in process
viseron3 | self.face_recognition(
viseron3 | File "/src/viseron/components/deepstack/face_recognition.py", line 88, in face_recognition
viseron3 | detection["box"]["x_min"] + x1,
viseron3 | KeyError: 'box'
viseron3 | [31m[2024-12-16 11:42:19] [ERROR ] [viseron.watchdog.thread_watchdog] - Thread viseron.domains.post_processor is dead, restarting[0m
viseron3 | [31m[2024-12-16 11:42:19] [ERROR ] [root] - Uncaught thread exception in thread viseron.domains.post_processor[0m
viseron3 | Traceback (most recent call last):
viseron3 | File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
viseron3 | self.run()
viseron3 | File "/usr/lib/python3.10/threading.py", line 953, in run
viseron3 | self._target(*self._args, **self._kwargs)
viseron3 | File "/src/viseron/domains/post_processor/__init__.py", line 131, in post_process
viseron3 | self.process(
viseron3 | File "/src/viseron/domains/face_recognition/__init__.py", line 136, in process
viseron3 | self.face_recognition(
viseron3 | File "/src/viseron/components/deepstack/face_recognition.py", line 88, in face_recognition
viseron3 | detection["box"]["x_min"] + x1,
viseron3 | KeyError: 'box'
viseron3 | [31m[2024-12-16 11:42:34] [ERROR ] [viseron.watchdog.thread_watchdog] - Thread viseron.domains.post_processor is dead, restarting[0m
viseron3 | [31m[2024-12-16 11:42:34] [ERROR ] [root] - Uncaught thread exception in thread viseron.domains.post_processor[0m
viseron3 | Traceback (most recent call last):
viseron3 | File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
viseron3 | self.run()
viseron3 | File "/usr/lib/python3.10/threading.py", line 953, in run
viseron3 | self._target(*self._args, **self._kwargs)
viseron3 | File "/src/viseron/domains/post_processor/__init__.py", line 131, in post_process
viseron3 | self.process(
viseron3 | File "/src/viseron/domains/face_recognition/__init__.py", line 136, in process
viseron3 | self.face_recognition(
viseron3 | File "/src/viseron/components/deepstack/face_recognition.py", line 88, in face_recognition
viseron3 | detection["box"]["x_min"] + x1,
viseron3 | KeyError: 'box'
I found the issue, The response coming from DeepStack is without box property.
Response
{
"success": true,
"predictions": [
{
"confidence": 0.8262921,
"userid": "pratik",
"y_min": 481,
"x_min": 378,
"y_max": 1806,
"x_max": 1321
},
{
"confidence": 0.73627937,
"userid": "papa",
"y_min": 1216,
"x_min": 1568,
"y_max": 2246,
"x_max": 2372
}
],
"duration": 0
}
I have done the below changes locally in /src/viseron/components/deepstack/face_recognition.py and seems to be working fine now
class FaceRecognition(AbstractFaceRecognition):
"""DeepSTack face recognition processor."""
def __init__(self, vis: Viseron, config, camera_identifier) -> None:
super().__init__(
vis, COMPONENT, config[CONFIG_FACE_RECOGNITION], camera_identifier
)
self._ds_config = config
self._ds = DeepstackFace(
ip=config[CONFIG_HOST],
port=config[CONFIG_PORT],
api_key=config[CONFIG_API_KEY],
timeout=config[CONFIG_TIMEOUT],
min_confidence=config[CONFIG_FACE_RECOGNITION][CONFIG_MIN_CONFIDENCE],
)
def face_recognition(
self, shared_frame: SharedFrame, detected_object: DetectedObject
) -> None:
"""Perform face recognition."""
frame = self._camera.shared_frames.get_decoded_frame_rgb(shared_frame)
x1, y1, x2, y2 = calculate_absolute_coords(
(
detected_object.rel_x1,
detected_object.rel_y1,
detected_object.rel_x2,
detected_object.rel_y2,
),
self._camera.resolution,
)
cropped_frame = frame[y1:y2, x1:x2].copy()
try:
detections = self._ds.recognize(
cv2.imencode(".jpg", cropped_frame)[1].tobytes()
)
except ds.DeepstackException as error:
self._logger.error("Error calling deepstack: %s", error)
for detection in detections:
if detection["userid"] != "unknown":
self._logger.debug(f"Face found: {detection}")
self.known_face_found(
detection["userid"],
(
detection["x_min"] + x1,
detection["y_min"] + y1,
detection["x_max"] + x2,
detection["y_max"] + y2,
),
shared_frame,
confidence=detection["confidence"],
)
else:
self.unknown_face_found(
(
detection["x_min"] + x1,
detection["y_min"] + y1,
detection["x_max"] + x2,
detection["y_max"] + y2,
),
shared_frame,
confidence=detection["confidence"],
)
Thanks for finding the issue, will fix it in dev soon.
Strange tho, it has definitely worked before and DeepStack has not been updated in a long time
Sorry totally missed the questions.
- How can I save the recordings in different tiers like 4 days in one folder path and older than that till 10 days in another folder path?
You configure it like this:
storage:
recorder:
tiers:
- path: /data
events:
max_age:
days: 4
- path: /data2
events:
max_age:
days: 14 # Add 10 to the first tiers `max_age`
- Are there ways In Timeline or any plans to add a feature, where we can export the mp4 file from the selected Start Time to the selected End Time?
Yes i am working on that as we speak!
- Is there any way, how we can access thumbnails http://192.168.2.109:8080/files/data/thumbnails/tapocam01/121.jpg, the same way we can access recordings http://192.168.2.109:8080/files/data/recordings/tapocam01/06:51:52.mp4 ?
What do you mean here exactly? The URL looks fine to me
Hi @roflcoopter , thanks for your reply! I absolutely love what you’ve created and would be thrilled to see it become even more amazing.
Is there any way, how we can access thumbnails http://192.168.2.109:8080/files/data/thumbnails/tapocam01/121.jpg, the same way we can access recordings http://192.168.2.109:8080/files/data/recordings/tapocam01/06:51:52.mp4 ?
The problem with this is, when I am accessing the mp4 files, I’m encountering a 404 error and noticed some strange behaviour while trying to find a workaround:
- When I placed a .mp4 file in /data/thumbnails/tapocam01/test.mp4, it wasn't accessible. However, when I changed the file extension to .jpg, it became accessible via the URL. Is there a filter in place that only allows access to .jpg files and not .mp4 files?
- In the Recordings folder, neither .mp4 nor .jpg files were accessible. It seems like routing might only be configured for the Thumbnails folder, not the Recordings folder.
I know the above might seem trivial, but I just want to send a notification in Home Assistant with the thumbnail and video URL, which, when clicked, will open the thumbnail and video.
Glad you like it! v3 is a huge bump in functionality for Viseron, hence why it has taken me so long.
Since you have create_event_clip: true the file should end up in /data/recordings/tapocam01/06:51:52.mp4
Could you enable debug logging and send me the logs from when the recording ends and the video is created?
logger:
default_level: debug
Hi @roflcoopter The Recordings are saved, as you mentioned in /data/recordings/tapocam01/06:51:52.mp4. Is it possible to access recorded video files stored in /data/recordings/tapocam01/ via a URL or web application?
Additionally, I'd like to create a URL that navigates to a specific point in the timeline for a particular camera. For example: Current URL: http://192.168.2.109:8080/#/events?date=2025-01-25&tab=timeline Desired URL: http://192.168.2.109:8080/#/events?date=2025-01-25&tab=timeline&time=14:03:02
This would allow me to include links in motion detection notifications that direct users to the relevant recording or timeline view. Any guidance or suggestions would be greatly appreciated!
Hi @roflcoopter The Recordings are saved, as you mentioned in /data/recordings/tapocam01/06:51:52.mp4. Is it possible to access recorded video files stored in /data/recordings/tapocam01/ via a URL or web application?
The mp4 files are no longer exposed through the API sadly. Depending on the client you could perhaps use the HLS URL that Viseron is using to play back the recordings in the interface.
Additionally, I'd like to create a URL that navigates to a specific point in the timeline for a particular camera. For example: Current URL: http://192.168.2.109:8080/#/events?date=2025-01-25&tab=timeline Desired URL: http://192.168.2.109:8080/#/events?date=2025-01-25&tab=timeline&time=14:03:02
This would allow me to include links in motion detection notifications that direct users to the relevant recording or timeline view. Any guidance or suggestions would be greatly appreciated!
That is not possible currently but it could definitely be added.
Could you open a separate issue for that so it can be tracked properly?
Hi,
I'm using 3.1.2 and encounter that same error for Deepstack Face Recognition:
Traceback (most recent call last):
File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
self.run()
File "/usr/lib/python3.10/threading.py", line 953, in run
self._target(*self._args, **self._kwargs)
File "/src/viseron/domains/post_processor/__init__.py", line 197, in post_process
_process(detected_objects_data, filtered_objects)
File "/src/viseron/domains/post_processor/__init__.py", line 167, in _process
self.process(
File "/src/viseron/domains/face_recognition/__init__.py", line 141, in process
self.face_recognition(post_processor_frame, detected_object)
File "/src/viseron/components/deepstack/face_recognition.py", line 102, in face_recognition
detection["box"]["x_min"] + x1,
KeyError: 'box'
Fixed in dev, will take some time before the builds for the RPI are done