Add Camera Tuning Implementation (Tuning Interface, Zone/Mask Drawer Tool, Labels Editor, OSD Texts, and Miscellaneous UI)

[kaburagisec]

Background

When I started developing ONVIF components, I encountered a significant obstacle. ONVIF has many services, and one of the most important is Imaging. With the current Viseron implementation—where all configurations can only be changed through config.yaml—every time I wanted to adjust an Imaging parameter like brightness, contrast, sharpness, exposure, and so on, I would have to restart the Viseron instance to see the results.

This process was extremely cumbersome. Increasing the brightness by just 10% required me to:

1. Change the configuration file
2. Restart the service
3. Wait for the camera to initialize
4. Then see the changes.

This workflow was inefficient, made fine-tuning difficult, and became even more painful if the user had multiple cameras. This brightness change simply sends a request to the ONVIF device to make the desired changes. It's simple, and I want to see the changes directly within Viseron. That's why I introduce to Camera Tuning.

What is Camera Tuning?

Camera Tuning is an interface that functions as a configuration helper or configuration companion specifically for all components related to "Camera." This interface provides interactive UI elements connected to the REST API, allowing users to tune camera parameters directly through a visual interface—without manually editing config.yaml.

It's important to note that not all configuration keys and not all domains are supported by this tuning system. In this pull request, only certain keys and domains are supported.

[!NOTE] For now Camera Tuning only changes the config.yaml without changing the default configuration of the camera itself directly (for example ONVIF which will be implemented later).

Camera Tuning Implementation

Currently it only supports camera, object_detector, motion_detector, face_recognition, and license_plate_recognition domains. With implementation details in each domain as follows:

Domain	Camera Tuning Implementations
Camera	1. Video Transform (only flip/rotate for now) 2. Managing OSD (On-Screen Display) Texts (timestamp/custom), this will close #114 3. Miscellaneous config
Object Detector	1. Labels editor 2. Zone and Mask drawer tool, this will answer #1152 3. Zone Labels editor 4. Zone Name editor 5. Miscellaneous config
Motion Detector	1. Mask drawer tool 2. Miscellaneous config
Face Recognition	1. Labels editor 2. Mask drawer tool
License Plate Recognition	1. Labels editor 2. Mask drawer tool
Image Classification	No implementation yet
Protocol-related	No implementation yet
Notification-related	No implementation yet

Commits List

No	Commit	What happened?
1	Set Content-Type header to application/json for responses	It turns out that the REST API on the webserver component does not specify the content type in its response header.
2	Add Camera Tune API handler for managing camera-related components configurations	The main commit on the backend side of this PR.
3	Add comprehensive tests for Tune API handler functionality	Unit test for Camera Tune API
4	Add Camera Tuning interface with Zone/Mask drawer, Labels Editor, OSD Texts, Miscellaneous, etc	The main commit on the frontend side of this PR.
5	Adjust footer top margin and update license link	This is to fit the Camera Tuning page, and I also think the footer top margin is too high.
6	Adjust layout padding for more responsive in mobile	For non-fix container use style: {{ paddingX: {xs: 1, md: 2}, paddingY: 0.5 }} For fix container use style: {{ paddingX: {xs: 1, md: 2} }}
7	Adjust scrollbar height and width for more proper layout	The scrollbar is too wide, non-standard, and takes up space.
8	Adjust editorWidth to 100vw	It would be better for the editor to have a width of 100% rather than just 80% on both mobile and desktop.
9	Bump vite version 7.1.9 -> 7.2.4	Upgrade vite version to 7.2.4 which has bug fixes and performance improvements.
10	Update Typography color for camera selection messages for consistency	Updated text color for consistency only.
11	Remove border from default toast options	I think the border on the toast is unnecessary, removed for consistency.

Notes (dev):

1. Is there a way to make config.yaml directly affect the Viseron instance? Because most NVRs are immediately affected by configuration changes without requiring a restart. I haven't found a way.
2. Changes made to config.yaml in the API update logic will remove the previously commented lines in config.yaml.
3. Why is DEEPSTACK_LABELS the same as the /detectors/models/darknet/coco.names file? But the documentation explains the table itself? Should I just delete it in tune.py? It's too long.
4. What's the definitive way to get available_labels for Ultralytics YOLO? The documentation requires the command docker compose logs | grep "Labels". I don't think this is best practice and shouldn't be done. Because of this, Ultralytics YOLO will have difficulty tuning its labels and will ultimately use the fallback labels I defined in this PR.
5. For OSD text, users should be aware that font size and padding will always follow pixel size, so they must understand the size (size) of each camera's resolution to configure OSD text that best suits the camera they want to tune.
6. OSD text (its position and shape) will be immune to video transforms, because the config update logic always prioritizes video transforms over drawtext.
7. Can the codec in the video_filters recorder be automated? It would be a shame if it had to be hardcoded in the UI and would ultimately be coded to h264, even though many cameras already support h265.
8. It's a shame that the face_recognition and license_plate_recognition domains don't trigger recording events, and I believe the min_confidence field should also be applied to each camera, rather than a general implementation (all cameras). (This also makes the UI change to filter recordings by event recording type useless as of PR #1150 😆)
9. Although the type interface in the frontend provides available_labels, unfortunately I don't know how to get/set these labels for the face_recognition and license_plate_recognition domains.

Notes (docs):

1. There needs to be a documentation change regarding all components that have zones/masks to directly use the Camera Tuning feature instead of going to image-map.net. I really mean all.
2. There needs to be a documentation change regarding all components that have labels to directly use the Camera Tuning feature instead of looking for available labels by checking each file individually.
3. There needs to be a separate section in the documentation regarding how to use and the functionality of Camera Tuning.
4. There needs to be information in the documentation that Camera Tuning will only work by changing config.yaml, which means requiring a Viseron restart to see the effects after the Tuning Config change.
5. If there are additional implementations (in the future) regarding other configurations that can be tuned in Camera Tuning, ideally there should be some kind of guidance/standard for each type of configuration data. For example, booleans should use toggle, strings should use form, and so on.
6. In the FFmpeg documentation, it is written about the codec key which says: "Instruct FFmpeg to re-encode the video. This is needed to add the timestamp since video filters cannot be used with copy codec", but why is it not written in the Rotating video section that this is also required?
7. For Miscellaneous Config, they will always be rendered with default values ​​according to the documentation, so even if the key is not in config.yaml, the UI will always be there, and if there are no changes at all to the UI, then when saving the config it will not force changes in config.yaml (because it uses the default value). Conversely, if there are changes in the UI and saving the config, it will change config.yaml by adding the key if it did not previously exist in config.yaml.

TODO:

Created several TODO lists for this PR based on feedback:

• [x] Refactor the Tune API handling so that it is not too big/long by moving it to the viseron/components/webserver/api/v1/tuning folder
• [x] Improved the handling of _get_available_labels to handle custom models if the user doesn't use the default model. Also, improved the ability to handle custom labels if the label_path key is not the default. (this includes Ultralytics YOLO)
• [x] Removed the DEEPSTACK_LABELS (default) duplicate with the COCO-dataset.
• [ ] Refactor some things related to UI styling.
• [ ] Tested using the ruamel.yaml package so that comments in config.yaml are not lost when editing with Camera Tuning.

[netlify[bot]]

✅ Deploy Preview for viseron canceled.

Name	Link
🔨 Latest commit	42cc800d32cf9377c93bf54f283511e91163b1a2
🔍 Latest deploy log	https://app.netlify.com/projects/viseron/deploys/6945b6d2f59fd30008388664

[kaburagisec]

It's very strange because on my side everything passes (except for codespell which turned out to be ignored) like this:

abc@f7899f18bc9e:/workspaces/viseron$ pre-commit run --all-files
pyupgrade................................................................Passed
autoflake................................................................Passed
isort....................................................................Passed
black....................................................................Passed
codespell................................................................Failed
- hook id: codespell
- exit code: 65

frontend/tests/setupTests.ts:4: afterAll ==> after all
frontend/tests/setupTests.ts:20: afterAll ==> after all

flake8...................................................................Passed
pylint...................................................................Passed
mypy.....................................................................Passed

[roflcoopter]

Answered most of your questions on Discord but ill do the same here for clarity:

1. Is there a way to make config.yaml directly affect the Viseron instance? Because most NVRs are immediately affected by configuration changes without requiring a restart. I haven't found a way.

There is no easy way no. When a camera changes, all other components/domains that rely on it would need to reload as well. Each component/domain would have to implement that functionality in order to properly stop and restart everything so we dont end up with zombie threads and processes

2. Changes made to config.yaml in the API update logic will remove the previously commented lines in config.yaml.

One option to remedy this is to migrate to ruamel: https://pypi.org/project/ruamel.yaml/

3. Why is DEEPSTACK_LABELS the same as the /detectors/models/darknet/coco.names file? But the documentation explains the table itself? Should I just delete it in tune.py? It's too long.

Deepstack labels are stored on the server, not in the container. It is trained on the COCO-dataset tho so thats why it uses the same as darknet.

4. What's the definitive way to get available_labels for Ultralytics YOLO? The documentation requires the command docker compose logs | grep "Labels". I don't think this is best practice and shouldn't be done. Because of this, Ultralytics YOLO will have difficulty tuning its labels and will ultimately use the fallback labels I defined in this PR.

Seems they are embedded in the models but they are also present in the python install dir: /usr/local/lib/python3.10/dist-packages/ultralytics/cfg/datasets/coco8.yaml

7. Can the codec in the video_filters recorder be automated? It would be a shame if it had to be hardcoded in the UI and would ultimately be coded to h264, even though many cameras already support h265.

Can you elaborate a little with what you mean here?

8. It's a shame that the face_recognition and license_plate_recognition domains don't trigger recording events, and I believe the min_confidence field should also be applied to each camera, rather than a general implementation (all cameras). (This also makes the UI change to filter recordings by event recording type useless as of PR Several UI/UX Improvements #1150 😆)

With the current architecture it could be hard to achieve that, but not impossible.

And regarding the docs, will you write anything there or do you want med to handle that stuff?

[john-]

4. What's the definitive way to get available_labels for Ultralytics YOLO? The documentation requires the command docker compose logs | grep "Labels". I don't think this is best practice and shouldn't be done. Because of this, Ultralytics YOLO will have difficulty tuning its labels and will ultimately use the fallback labels I defined in this PR.

Seems they are embedded in the models but they are also present in the python install dir: /usr/local/lib/python3.10/dist-packages/ultralytics/cfg/datasets/coco8.yaml

Models can be trained with labels that differ from the stock model (coco8). For example, in my application I have only 8 or so of the coco labels and another 6 that do not appear in coco dataset (e.g. Opossum).

In the deployed state, these would only appear in the model.

The steps in the YOLO documentation are for end users to get easy access to the arbitrary labels used in the particular model they have deployed.

There is a way using the Python API (maybe other ways) to get the list of labels. Is that what would be needed to avoid fallback to fixed (and possibly incorrect) set of labels?

I have not read all the thread yet so if I am missing context let me know.

[kaburagisec]

Answered most of your questions on Discord but ill do the same here for clarity:

1. Is there a way to make config.yaml directly affect the Viseron instance? Because most NVRs are immediately affected by configuration changes without requiring a restart. I haven't found a way.

There is no easy way no. When a camera changes, all other components/domains that rely on it would need to reload as well. Each component/domain would have to implement that functionality in order to properly stop and restart everything so we dont end up with zombie threads and processes

OK understood, no problem, but still later the ONVIF implementation will directly change the configuration directly because it only requires an API for that.

2. Changes made to config.yaml in the API update logic will remove the previously commented lines in config.yaml.

One option to remedy this is to migrate to ruamel: https://pypi.org/project/ruamel.yaml/

Initially I implemented the Tune API with ruamel.yaml, but this function only works for older versions of ruamel.yaml, namely versions below 0.18.0, as quoted from the official page (https://yaml.dev/doc/ruamel-yaml/):

" As announced, in 0.18.0, the old PyYAML functions have been deprecated. (scan, parse, compose, load, emit, serialize, dump and their variants (all, safe, round_trip, etc)). If you only read this after your program has stopped working: I am sorry to hear that, but that also means you, or the person developing your program, has not tested with warnings on (which is the recommendation in PEP 565, and e.g. defaulting when using pytest). If you have troubles, explicitly use

pip install "ruamel.yaml<0.18.0"

or put something to that effects in your requirments, to give yourself some time to solve the issue. "

According to the statement above, if we use ruamel.yaml, we have to pin the version to 0.17.40, which was released in 2023.

Personally, I avoid using older versions of a library for one or two reasons, including security issues. But if there's no problem, I'll implement it. What do you think?

3. Why is DEEPSTACK_LABELS the same as the /detectors/models/darknet/coco.names file? But the documentation explains the table itself? Should I just delete it in tune.py? It's too long.

Deepstack labels are stored on the server, not in the container. It is trained on the COCO-dataset tho so thats why it uses the same as darknet.

Okk means I will remove DEEPSTACK_LABELS in API, because it duplicates with COCO-dataset.

4. What's the definitive way to get available_labels for Ultralytics YOLO? The documentation requires the command docker compose logs | grep "Labels". I don't think this is best practice and shouldn't be done. Because of this, Ultralytics YOLO will have difficulty tuning its labels and will ultimately use the fallback labels I defined in this PR.

Seems they are embedded in the models but they are also present in the python install dir: /usr/local/lib/python3.10/dist-packages/ultralytics/cfg/datasets/coco8.yaml

Ok, It seems that contributor @john- has joined the discussion, I will ask him further.

7. Can the codec in the video_filters recorder be automated? It would be a shame if it had to be hardcoded in the UI and would ultimately be coded to h264, even though many cameras already support h265.

Can you elaborate a little with what you mean here?

In the section https://viseron.netlify.app/components-explorer/components/ffmpeg#adding-timestamp-to-video, it states that the key codec must be present in the recorder, because: Instruct FFmpeg to re-encode the video. This is needed to add the timestamp since video filters cannot be used with copy codec

Since the codec key must always be present on the recorder, I ended up defining the codec as always h264 on the line https://github.com/roflcoopter/viseron/pull/1165/files#diff-8e9ee154e70a830a7fd7675037e8c13e765209b2ad59ac2ddbcdacfc219db561R7, but shouldn't this codec be automatically obtained from FFprobe? Or am I misunderstanding this?

8. It's a shame that the face_recognition and license_plate_recognition domains don't trigger recording events, and I believe the min_confidence field should also be applied to each camera, rather than a general implementation (all cameras). (This also makes the UI change to filter recordings by event recording type useless as of PR Several UI/UX Improvements #1150 😆)

With the current architecture it could be hard to achieve that, but not impossible.

Ok, maybe I can explore this further in future iterations.

And regarding the docs, will you write anything there or do you want med to handle that stuff?

I want to write in docs, but unfortunately I'm afraid that my writing style is different from yours, so I won't write the docs.

[kaburagisec]

There is a way using the Python API (maybe other ways) to get the list of labels. Is that what would be needed to avoid fallback to fixed (and possibly incorrect) set of labels?

I have not read all the thread yet so if I am missing context let me know.

So, this PR allows us to manage the labels applied to the object_detector component directly through the UI. Since all existing object_detector components have clear instructions for getting available labels except Ultralytics YOLO, I'm asking for a definitive way to get labels in the Ultralytics YOLO component.

Could you please tell me a definitive way to get labels based on the user's selected model other than using the command: docker compose logs | grep "Labels" ?

Thank you

[john-]

Could you please tell me a definitive way to get labels based on the user's selected model other than using the command: docker compose logs | grep "Labels" ?

Thank you

I will take a look at this.

[kaburagisec]

@john- Thank You!

This is what this PR looks like when rendered, so labels can be edited directly from the UI, but requires available_labels to work properly:

[john-]

In order to get my bearings I am going over existing documenation for other components to see how they handle model specific labels.

Since all existing object_detector components have clear instructions for getting available labels

Can you show me what you mean?

DeepStack has this:

Labels are used to tell Viseron what objects to look for and keep recordings of. The available labels depends on what detection model you are using.

They list the the default labels that may or may not be what the model contains.

Codeproject.AI says the same thing.

Darknet does have a config item label_path. This is optional.

[kaburagisec]

For the darknet, label_path is directly used. See the code line https://github.com/roflcoopter/viseron/pull/1165/files#diff-ca47ebe269d7b6c79b186081e10b6229a1de12bbc9e820f0671f81b109239027R217.

For the code project, there is also model-based handling. See the line https://github.com/roflcoopter/viseron/pull/1165/files#diff-ca47ebe269d7b6c79b186081e10b6229a1de12bbc9e820f0671f81b109239027R224 and https://github.com/roflcoopter/viseron/pull/1165/files#diff-ca47ebe269d7b6c79b186081e10b6229a1de12bbc9e820f0671f81b109239027R17

For Deepstack, the current implementation for available_labels with the default model. See the code at https://github.com/roflcoopter/viseron/pull/1165/files#diff-ca47ebe269d7b6c79b186081e10b6229a1de12bbc9e820f0671f81b109239027R71.

So, can you give me some guidance on Ultralytics YOLO?

[john-]

I reviewed the links you provided but they took me to the top of an 80 file commit. I searched (ctrl-f in browser) but nothing got me closer to understanding how these components address this.

This is probably not what you are looking for as I expect the proper solution (just a guess) is to extend the YOLO component:

from ultralytics import YOLO

# Load a model
model = YOLO("my_custom_model.pt")

print(model.names)

output:

{0: 'bicycle', 1: 'bird', 2: 'bus', 3: 'car', 4: 'cat', 5: 'dog', 6: 'motorcycle', 7: 'person', 8: 'truck',9: 'squirrel', 10: 'car-light', 11: 'rabbit', 12: 'fox', 13: 'opossum', 14: 'skunk', 15: 'racoon', 16: 'spider'}

[kaburagisec]

I reviewed the links you provided but they took me to the top of an 80 file commit. I searched (ctrl-f in browser) but nothing got me closer to understanding how these components address this.

This is probably not what you are looking for as I expect the proper solution (just a guess) is to extend the YOLO component:

from ultralytics import YOLO

# Load a model
model = YOLO("my_custom_model.pt")

print(model.names)

output:

{0: 'bicycle', 1: 'bird', 2: 'bus', 3: 'car', 4: 'cat', 5: 'dog', 6: 'motorcycle', 7: 'person', 8: 'truck',9: 'squirrel', 10: 'car-light', 11: 'rabbit', 12: 'fox', 13: 'opossum', 14: 'skunk', 15: 'racoon', 16: 'spider'}

That's right, this is what I meant, thank you, it means the implementation of available_labels for Ultralytics YOLO is by taking the model_path​ value from config.yaml and accessing it through the YOLO Class as you demonstrated above.

[john-]

Sounds good. Let me know if you need anything else. I am looking forward to all the improvements you have been putting into Viseron!

[kaburagisec]

FYI, the link I included above is a link to highlight certain lines in the commit, the highlight will be yellow like this:

Example: https://github.com/roflcoopter/viseron/pull/1165/files#diff-ca47ebe269d7b6c79b186081e10b6229a1de12bbc9e820f0671f81b109239027R17

Highlighted:

[kaburagisec]

Created several TODO lists for this PR based on feedback:

• [x] Refactor the Tune API handling so that it is not too big/long by moving it to the viseron/components/webserver/api/v1/tuning folder
• [x] Improved the handling of _get_available_labels to handle custom models if the user doesn't use the default model. Also, improved the ability to handle custom labels if the label_path key is not the default. (this includes Ultralytics YOLO)
• [x] Removed the DEEPSTACK_LABELS (default) duplicate with the COCO-dataset.
• [x] Tested using the ruamel.yaml package so that comments in config.yaml are not lost when editing with Camera Tuning.

[roflcoopter]

Thanks for chipping in @john- !

Regarding ruamel, what I meant is that we could switch to ruamel completely and remove the PyYAML dependency

[kaburagisec]

Regarding ruamel, what I meant is that we could switch to ruamel completely and remove the PyYAML dependency

Okay then, but are you sure we're not using PyYAML at all?

[kaburagisec]

1. https://github.com/roflcoopter/viseron/pull/1165/commits/7cbbbcd953f3bda70ea9cc5260c21d7f518b7ef1 : This commit splits the Camera Tuning API functionality by domain by moving it to the tuning folder. So that it is easier to read and maintain.
2. https://github.com/roflcoopter/viseron/pull/1165/commits/c00716c9d027a8d734904c397e68be91c9be2d54 : This commit changes the logic for dynamically retrieving available_labels from an object_detector component if custom models/labels are available. If this fails, available_labels will not be displayed in the API response.
3. https://github.com/roflcoopter/viseron/pull/1165/commits/71594cfae32716316cd551a03054466aa5ec2466 : This commit is to accommodate the above change on the frontend side, so if available_labels is not available on the API response, instead of using select control with the available_labels fallback defined, the dialog label will be a regular text input.

Important Notes:

• The Ultralytics YOLO component states that: "Models are not installed by default. See below for steps to define the model as well as make them available to Viseron.", but you did not change the model_path to required, instead you defined it as optional with the default value: /detectors/models/yolo/default.pt which does not exist. This will be very ambiguous for users, could you please take a look? @john-

[john-]

The Ultralytics YOLO component states that: "Models are not installed by default. See below for steps to define the model as well as make them available to Viseron.", but you did not change the model_path to required, instead you defined it as optional with the default value: /detectors/models/yolo/default.pt which does not exist. This will be very ambiguous for users, could you please take a look? @john-

Yes I will address this.

[kaburagisec]

No CI jobs failed, all issues were resolved, comments remained in config.yaml when updating the config, YAML tags like !secret were preserved, and external font dependencies were removed.

I believe this PR is sufficient and can serve as a starting point for configuring Viseron directly in the UI. But please note that Camera Tuning only applies to config keys located under <CAMERA_IDENTIFIER>.