Viseron v2

[roflcoopter]

This is a huge rewrite of Viseron. It focuses on decoupling all parts of Viseron, making it more modularized. This allows for easier integration of new functionality.

The config.yaml file will change completely so there is some work that has to be done by each user to port over to the new version. The general config format is a component which implements one or more domains. Each camera has a unique camera identifier which flows through the entire configuration.

The nvr component then ties all these different domains together and provides the full functionality.

The big benefit of this new format is that you can mix and match components more freely. For instance, you could use different object detectors for different cameras, you are not tied into just one.

Config Example

ffmpeg: # <-- component
  camera: # <-- domain
    camera_one: # <-- camera identifier
       name: Camera 1
       host: 192.168.10.10
       username: test
       password: test
    camera_two: # <-- camera identifier
       name: Camera 2
       host: 192.168.10.11
       username: test
       password: test
       ....

darknet: # <-- component
  object_detector: # <-- domain
    model: /my_custom_model/model.weights
    cameras:
      camera_one: # <-- camera identifier
        fps: 5

deepstack:  # <-- component
  host: deepstack # <-- component config option
  port: 5000 # <-- component config option
  object_detector: # <-- domain
    cameras:
      camera_two:  # <-- camera identifier
        fps: 1
        labels:
          - label: person
            confidence: 0.75
            trigger_recorder: false
  face_recognition: # <-- domain
    cameras:
      camera_one:
      camera_two:
    labels:
      - person

background_subtractor: # <-- component
  motion_detector: # <-- domain
    cameras:
      camera_one: # <-- camera identifier
        fps: 1
        mask:
          - coordinates:
              - x: 400
                y: 200
              - x: 1000
                y: 200
              - x: 1000
                y: 750
              - x: 400
                y: 750

nvr: # <-- component
  camera_one: # <-- camera identifier
  camera_two: # <-- camera identifier

Here is a generic config snippet with publicly available cameras that can be used to try it out

ffmpeg:
  camera:
    viseron_camera:
      name: Camera 1
      host: 195.196.36.242
      path: /mjpg/video.mjpg
      port: 80
      stream_format: mjpeg
      fps: 6
      recorder:
        idle_timeout: 1
        codec: h264
    viseron_camera2:
      name: Camera 2
      host: storatorg.halmstad.se
      path: /mjpg/video.mjpg
      stream_format: mjpeg
      port: 443
      fps: 2
      protocol: https
      recorder:
        idle_timeout: 1
        codec: h264
    viseron_camera3:
      name: Camera 3
      host: 195.196.36.242
      path: /mjpg/video.mjpg
      port: 80
      stream_format: mjpeg
      fps: 6
      recorder:
        idle_timeout: 1
        codec: h264

mog2:
  motion_detector:
    cameras:
      viseron_camera:
        fps: 1

background_subtractor:
  motion_detector:
    cameras:
      viseron_camera3:
        fps: 1
        mask:
          - coordinates:
              - x: 400
                y: 200
              - x: 1000
                y: 200
              - x: 1000
                y: 750
              - x: 400
                y: 750

darknet:
  object_detector:
    model_path: /detectors/models/darknet/yolov4-csp-x-swish.weights
    model_config: /detectors/models/darknet/yolov4-csp-x-swish.cfg
    cameras:
      viseron_camera:
        fps: 1
        scan_on_motion_only: false
        labels:
          - label: person
            confidence: 0.2
            trigger_recorder: false
      viseron_camera2:
        fps: 1
        labels:
          - label: person
            confidence: 0.2
            trigger_recorder: false

nvr:
  viseron_camera:
  viseron_camera2:
  viseron_camera3:

webserver:

mqtt:
  broker: ****
  port: 1883
  username: ****
  password: *****
  client_id: viseron
  home_assistant:

logger:
  default_level: debug
  logs:
    viseron.components.data_stream: info
    viseron.components.background_subtractor: info
    viseron.components.mog2: info

End goal is to port all existing functionality and also resolve some of the open issues.

This PR and its description will be updated going forward.

To-do list:

• [x] EdgeTPU Object detector
• [x] Darknet Object detector
• [x] DeepStack Object detector
• [x] MoG2 Motion detector
• [x] Background subtractor Motion detector
• [x] Webserver
• [ ] MQTT
  • [x] Client
  • [ ] Home Assistant integration
    • [x] Camera status binary sensors
    • [x] Motion detected binary sensors
    • [x] Object detected binary sensors
    • [x] Face detected binary sensors
    • [x] Sensors
    • [x] Switches
    • [ ] Cameras
• [x] FFmpeg camera
• [x] Recordings
• [x] Deepstack Face recognition
• [x] OpenCV Face recognition
• [x] Internal Entity registry that components can listen and react to (inspired by Home Assistant)
• [x] Setup retry functionality of components
• [x] React based UI
• [x] Letterbox resizing before object detection
• [x] Default config generation
• [ ] Remove all old and dead code
• [ ] Update tests
• [ ] Update documentation

Issues to resolve

• [x] #300
• [x] #297
• [x] #294
• [x] #304
• [x] #307
• [x] #309
• [x] #145
• [x] #284
• [x] #311
• [x] #312
• [x] #313
• [x] #326
• [x] #289
• [x] #341
• [ ] #345

Bonus features not possible with previous versions of Viseron

• Multiprocessing
• Using different object or motion detectors simultaneously

Partial changelog (README has not been updated to reflect this)

Breaking changes

• ALL kinds of inheritance in the config has been removed. This means that you have to explicitly have to configure your object detector and motion detector settings for each nvr.
• interval has been removed from object_detection and motion_detection A new config option fps will be used instead. Please update your configuration. This change was made since it was quite confusing, both in the code and for the users because interval was specified in seconds.
• logging has been removed in all shapes and forms and has been replaced with logger Please see the updated documentation
• cameras config section has been removed. Camera config is now specified under a component.
• Each object detector has been split up into individual components. See the documentation for each detector.
• Each motion detector has been split up into individual components. See the documentation for each detector.
• recorder can no longer be configured on a global level. It now has to be present under each camera configuration.
• timeout under recorder is now called idle_timeout
• static_mjpeg_streams are now called mjpeg_streams
• enable under object_detection is no more. To disable object detection you simply dont configure it for a camera Same goes for motion_detector
• timeout for motion_detector is now called recorder_keepalive
• max_timeout for motion_detector is now called max_recorder_keepalive
• Recordings are now stored in the folder structure /recordings/<camera name>/<date>/<timestamp>.mp4
• filter_args removed for camera and recorder. For camera, use video_filters instead. For recorder, you can use both video_filters and audio_filters
  Short config example to rotate video 180 degrees:

  ffmpeg:
    camera:
      camera_1:
      ....
        video_filters:   # These filters rotate the images processed by Viseron
          - transpose=2
          - transpose=2
        recorder:
          video_filters:   # These filters rotate the recorded video
            - transpose=2
            - transpose=2
  

New features

• New config entry for webserver. See the README for available config options
• React based Web UI
• New component gstreamer(!) The implementation is quite basic as of now, but this will pave way for a better Jetson Nano integration.
• New post_processor, image_classification. Right now the edgetpu component has support for it.

Changes

Fixes

• Fixed detection of usb EdgeTPU, so that permissions are set correctly

[Shameerpv]

Hi. appreciate your efforts . I was trying out version 2 . Can you share a sample file for new config.yaml . I am looking for config section for ""nvr" component,

thanks in advance

[fathome]

Thanks alot for all the effort. About the jetson nano problems. I've found this: https://github.com/timongentzsch/Jetson_Ubuntu20_Images . I still have to run tests, but it might make it possible to make python 3.8 work without having to compile it. Besides that, gstreamer support would also be great. There is a pull request for frigate nvr that replaces ffmpeg with gstreamer for the nano (https://github.com/blakeblackshear/frigate/pull/2548). Docker images should be based on the latest l4t version, Not sure if all balenalib images are using the latest.

[roflcoopter]

Hi. appreciate your efforts . I was trying out version 2 . Can you share a sample file for new config.yaml . I am looking for config section for ""nvr" component,

thanks in advance

Cool! Its quite simple really, you just have to supply the camera identifier, something like this:

ffmpeg:
  camera:
    my_camera_identifier:
       host: 192.168.0.10
       ....

nvr:
  my_camera_identifier:

[roflcoopter]

Thanks alot for all the effort. About the jetson nano problems. I've found this: https://github.com/timongentzsch/Jetson_Ubuntu20_Images . I still have to run tests, but it might make it possible to make python 3.8 work without having to compile it. Besides that, gstreamer support would also be great. There is a pull request for frigate nvr that replaces ffmpeg with gstreamer for the nano (blakeblackshear/frigate#2548). Docker images should be based on the latest l4t version, Not sure if all balenalib images are using the latest.

Awesome, will take a look at those when i get to the Nano stuff. Will see if ill be able to implement gstreamer before the release, shouldnt be too hard since there are some boilerplate code i could use in that PR

[John237-y]

Awesome work. I am struggling with the configurations for new version. Seems like config file is changed completely . It would be beneficial if you could share the updated configuration format.

[roflcoopter]

Awesome work. I am struggling with the configurations for new version. Seems like config file is changed completely . It would be beneficial if you could share the updated configuration format.

Just updated the description with som explanations, not sure if its enough information tho. I will soon do the work of updating the entire documentation, which will probably take a while since there is a lot to cover!

[gniknalu]

I want to try it on one of my other hosts but not sure how to pull the docker container. My apologies for asking such a basic question.

[roflcoopter]

I want to try it on one of my other hosts but not sure how to pull the docker container. My apologies for asking such a basic question.

You would have to build it manually for now since its not released yet. There are still a lot of moving parts and the documentation is not up to par quite yet, so i suggest you wait a little longer!

[roflcoopter]

Got a little sneakpeak for you today! Been working hard on the UI, which is the reason i havent pushed any commits in a while.

The UI features:

• A responsive design
• Material UI
• Dark and light mode

What do you think?

See the screenshots below Cameras: Recordings:

And a GIF demo (low quality

[Shameerpv]

Got a little sneakpeak for you today! Been working hard on the UI, which is the reason i havent pushed any commits in a while.

The UI features:

• A responsive design
• Material UI
• Dark and light mode

What do you think?

See the screenshots below Cameras: Recordings:

And a GIF demo (low quality

New UI looks awesome!! . Did you commit the changes ? I would love to try it.

[roflcoopter]

@Shameerpv Not yet, most likely tomorrow.

I will then set up an alpha build so you all can try it out without building manually

[jamitupya]

Looking very good :) look forward to having a go at a build.

[roflcoopter]

All commits are now pushed, but you need to build manually to test for now.

Here is a generic config snippet that i used to provide the demo. It demonstrates the new configuration format pretty well

ffmpeg:
  camera:
    viseron_camera:
      name: Camera 1
      host: 195.196.36.242
      path: /mjpg/video.mjpg
      port: 80
      stream_format: mjpeg
      fps: 6
      recorder:
        idle_timeout: 1
        codec: h264
    viseron_camera2:
      name: Camera 2
      host: storatorg.halmstad.se
      path: /mjpg/video.mjpg
      stream_format: mjpeg
      port: 443
      fps: 2
      protocol: https
      recorder:
        idle_timeout: 1
        codec: h264
    viseron_camera3:
      name: Camera 3
      host: 195.196.36.242
      path: /mjpg/video.mjpg
      port: 80
      stream_format: mjpeg
      fps: 6
      recorder:
        idle_timeout: 1
        codec: h264

mog2:
  motion_detector:
    cameras:
      viseron_camera:
        fps: 1

background_subtractor:
  motion_detector:
    cameras:
      viseron_camera3:
        fps: 1
        mask:
          - coordinates:
              - x: 400
                y: 200
              - x: 1000
                y: 200
              - x: 1000
                y: 750
              - x: 400
                y: 750

darknet:
  object_detector:
    model_path: /detectors/models/darknet/yolov4-csp-x-swish.weights
    model_config: /detectors/models/darknet/yolov4-csp-x-swish.cfg
    cameras:
      viseron_camera:
        fps: 1
        scan_on_motion_only: false
        labels:
          - label: person
            confidence: 0.2
            trigger_recorder: false
      viseron_camera2:
        fps: 1
        labels:
          - label: person
            confidence: 0.2
            trigger_recorder: false

deepstack:
  host: deepstack
  port: 5000
  object_detector:
    image_width: 608
    image_height: 608
    cameras:
      viseron_camera3:
        fps: 1
        labels:
          - label: person
            confidence: 0.2
            trigger_recorder: false

  face_recognition:
    save_unknown_faces: false
    cameras:
      viseron_camera:
      viseron_camera2:
      viseron_camera3:
    labels:
      - person

nvr:
  viseron_camera:
  viseron_camera2:
  viseron_camera3:

webserver:

mqtt:
  broker: ****
  port: 1883
  username: ****
  password: *****
  client_id: viseron
  home_assistant:

logger:
  default_level: debug
  logs:
    viseron.components.data_stream: info
    viseron.components.background_subtractor: info
    viseron.components.mog2: info

[gniknalu]

AWESOME!!!!!!

On Tue, Feb 8, 2022, 7:15 AM Jesper @.***> wrote:

Got a little sneakpeak for you today! Been working hard on the UI, which is the reason i havent pushed any commits in a while.

The UI features:

• A responsive design
• Material UI
• Dark and light mode

What do you think?

See the screenshots below Cameras: [image: Viseron-screenshot-cameras] https://user-images.githubusercontent.com/26493864/152993795-4f26735d-1cb2-4883-b765-bab8b3105a64.png Recordings: [image: Viseron-screenshot-recordings] https://user-images.githubusercontent.com/26493864/152993805-23660e01-4c69-4b29-b308-635bc777b35c.png

And a GIF demo (low quality [image: Alt Text] https://camo.githubusercontent.com/c4ddf65cff0bc7d58917b024f7fcf0cd01f3e6c19fe279d9c766618f8dabfb6b/68747470733a2f2f6d656469612e67697068792e636f6d2f6d656469612f52745553733150654831716242324f5155632f67697068792d646f776e73697a65642d6c617267652e676966

— Reply to this email directly, view it on GitHub https://github.com/roflcoopter/viseron/pull/306#issuecomment-1032596245, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA33ZSI5BBCSEH5KBTSVNODU2EJNRANCNFSM5JLOKHAQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you are subscribed to this thread.Message ID: @.***>

[gniknalu]

Got a little sneakpeak for you today! Been working hard on the UI, which is the reason i havent pushed any commits in a while.

The UI features:

• A responsive design
• Material UI
• Dark and light mode

What do you think?

I'm excited - looks great! Not only because of the UI but especially because of the individual configuration parameters for each camera. Will take my installation to the very next level. Wanting to also be able to showcase this to the HA community as this, in it's current state, is superior to the others out there. In this future version it will really stand out from the rest!

[jamitupya]

Agree, I look forward to getting into it.

Need to figure out how to do the manual build this arvo or tomorrow I hope :-) will grab a Jetson from storage and let you know how it goes.

[fathome]

very nice. i'am looking forward to testing this on the nano. i'am currently using frigate with the gstreamer pull request. as soon as this works on viseron, i'll give it a spin (i understand its not on the toaap of the list and thats fine). thank you again for all the work you put into that. gonna buy you a coffee

[roflcoopter]

Agree, I look forward to getting into it.

Need to figure out how to do the manual build this arvo or tomorrow I hope :-) will grab a Jetson from storage and let you know how it goes.

There are some instructions in the contributing guidelines on how to build, let me know if that isnt enough to get you started

very nice. i'am looking forward to testing this on the nano. i'am currently using frigate with the gstreamer pull request. as soon as this works on viseron, i'll give it a spin (i understand its not on the toaap of the list and thats fine). thank you again for all the work you put into that. gonna buy you a coffee

Im thinking i will look into the Jetson problems now as well as the documentation part. @fathome @jamitupya does any of you have the ability to setup a Nano on its own network and let med connect to it via VPN so i can run some tests? Would greatly speed up my efforts

[gniknalu]

Got a little sneakpeak for you today! Been working hard on the UI, which is the reason i havent pushed any commits in a while. The UI features:

• A responsive design
• Material UI
• Dark and light mode

What do you think?

Trying to figure out how to build it locally. . . .

[roflcoopter]

Got a little sneakpeak for you today! Been working hard on the UI, which is the reason i havent pushed any commits in a while. The UI features:

• A responsive design
• Material UI
• Dark and light mode

What do you think?

Trying to figure out how to build it locally. . . .

Am building it atm but i need to build FFmpeg and OpenCV which takes about 10hrs total(!) to complete.

You can see the progress of the builds in the Checks tab

[gniknalu]

Got a little sneakpeak for you today! Been working hard on the UI, which is the reason i havent pushed any commits in a while. The UI features:

• A responsive design
• Material UI
• Dark and light mode

What do you think?

Trying to figure out how to build it locally. . . .

Am building it atm but i need to build FFmpeg and OpenCV which takes about 10hrs total(!) to complete.

You can see the progress of the builds in the Checks tab

No worries - thank you. I'll monitor.

[jamitupya]

@roflcoopter - absolutely, however you'll have to bare with me this weekend. we are moving ISP's so need to sort that first. i also have an incredibly crap internet connection (20mb wifi) so theres that.

i'll setup with a camera on the same vlan if that makes it easier?

[roflcoopter]

@roflcoopter - absolutely, however you'll have to bare with me this weekend. we are moving ISP's so need to sort that first. i also have an incredibly crap internet connection (20mb wifi) so theres that.

i'll setup with a camera on the same vlan if that makes it easier?

Yeah no rush, that sounds great! Maybe you could set it up with ngrok? That would be the easiest since then i can simply SSH over

[roflcoopter]

These images have now been pushed and can be pulled for testing: roflcoopter/aarch64-viseron:modularize roflcoopter/amd64-viseron:modularize roflcoopter/amd64-cuda-viseron:modularize roflcoopter/rpi3-viseron:modularize

The Nano build failed for some reason sadly, ill let you know when those are available

[gniknalu]

These images have now been pushed and can be pulled for testing: roflcoopter/aarch64-viseron:modularize roflcoopter/amd64-viseron:modularize roflcoopter/amd64-cuda-viseron:modularize roflcoopter/rpi3-viseron:modularize

Exciting! I'll do a pull right now! Thank you!

[gniknalu]

modularize

Exciting! I'll do a pull right now! Thank you!

Where would you prefer we log problems - here? Or in "issues" or elsewhere?

Thank you

[roflcoopter]

Issues?! There cant possibly be issues?!

Jokes aside, its probably best that you create separate issues, easier to track that way

[gniknalu]

Issues?! There cant possibly be issues?!

Jokes aside, its probably best that you create separate issues, easier to track that way

I am seeing a repeating error that I don't understand. Making sure it's not config file related. Then I'll log them. Thank you again.

[gniknalu]

My alerts are working! My thumbnails are working! (They're not persistent between viserson stop/start though but that's OK) Webserver working! Looking forward to see the future work you have in store for this version!

Now I'm going to add all the other cameras . . . . .

[jamitupya]

@roflcoopter, i was just getting around to this but it appears storage container was damaged in a recent storm and it appears the Nanos are a victim of that.

Sad sad state of affairs, 22 Nano's DOA in the storage :S :-(

hopefully someone else has a spare one they can help with, if not let me know and i still have 1 on my desk but its being used for something for work currently.

cries in insurance claims