on-NVR video analytics

[scottlamb]

IMHO, this is superior to on-camera motion detection (#29) because from what I've seen on-camera motion detection kind of sucks:

• it has a bunch of false negatives and positives; there at least exist algorithms I could try on the NVR that are somewhat better
• it has completely different options between camera brands
• it's very tedious to tune because you can't just save a bunch of video segments and try out different options on them
• it's all a closed-source, mostly undocumented black box

on the other hand, on-NVR has more hardware needs (details TBD but significant work to do which scales with the number of cameras) and is much more complex to implement:

• needs more schema work as noted in #28 to track config (algorithm, parameters, mask), progress (last segment handled, reference image), and perhaps output (if more detail is desired than just the on/off/unknown signal; maybe a number indicating how much motion is in each grid square or something)
• needs a driver process that feeds it segments as they become available and probably skips some if it falls too far behind
• needs H.264/H.265 decoding—not too hard to use ffmpeg for this but might want to take advantage of hardware decoding
• needs some algorithm development, and there are a lot of possible algorithms (classical and machine learning!) out there with different pros and cons
• to make a decent algorithm testbed, needs H.264/H.265 encoding so you can create a video that shows you the reference image and motion detection mask alongside the source image
• might require using fancier instructions/hardware (SIMD, GPU, DSP, ISP, and/or TPU) for decent performance
• might want to support offloading to another machine via RPC boundary

[scottlamb]

Here's an interesting discussion to follow about using machine learning-based object detection with inexpensive NPUs such as the Coral USB Accelerator: https://github.com/ccrisan/motioneyeos/issues/1505

[xnorpx]

I wrote this for my blue iris server, https://github.com/xnorpx/blue-candle but it's far away from realtime so it works best with some simple classic motion detection and then run yolo to determine the actual object.