bi-aidetection
bi-aidetection copied to clipboard
VorlonCD
Feature request: Cropping images before processing.
I found this thread: https://forum.deepstack.cc/t/didnt-detect-the-cat/497
And it got me thinking, Today I'm using zones in BI for the motion sensing to reduce the number of images to be processed. Since I already have determined what part of the picture I'm interested in, would it not be logical to actually crop images before sending them to Deepstack?
The benefits would be smaller images and faster processing. Quicker processing would allow more frames to be processed, helping where the system is bogged down when many cameras trigger at once, or when higher bursts of images are needed to get a detection (like APR?).
Could it be implemented like the manual masking tool? Even a simple cropping-rectangle could easily save at least some performance.
Good idea, I can look into that. But be aware you loose a little quality every-time a JPG file is re-saved due to lossy compression.
Which reminds me that I think most of these object detection tools internally resize the image file to match the dimensions of the original training datasets BEFORE asking the AI what objects are in the image. For example, see the INPUT SIZE here: https://coral.ai/models/
And when I dug into some of the python code that deepstack uses I believe it does the same thing and the difference between LOW, MEDIUM and HIGH is how big the training dataset was and what it re-sizes the image to before it has the AI process it. And I believe I verified their image set size was also very small like less than 300x300 pixels.
Doods for example says "You should optimally pass image data in the requested size for the detector. If not, it will be automatically resized. It can read BMP, PNG and JPG as well as PPM. For detectors that do not specify a size (inception) you do not need to resize"
So maybe I'll look into resizing before sending. At the very least that will send less image data via http post. But then again I could be totally misunderstanding how it works behind the scenes since I havent had a chance to dig very much.
Interesting! So basically the processing time would not be affected by a smaller input image.
However, if the image is resized by Deepstack or Doods, and especially if it's to a quite small new size, could cropping the original image and thereby "enlarging" the objects of interest, increase the detection rate or probability-score?
Yes re-saving the image when working in a lossy format would degrade its quality, but for most of us, a still from a camera captured from a h.264/h.265 stream would not be that great to begin with and i don't expect much more detail to be lost. Another interesting aspect is how, or if, this would do any real difference for the AI as it's clearly not looking at the image and details the same way we do. But such speculation is a bit off topic. ;)
I keep starting and stopping working on this feature due to squirrels but its halfway there. Ability to change size, jpeg compression, brightness, contrast before sending to deepstack. I'll get it in eventually!
This is less of an issue for me right now. With the new GPU powered deepstack i can run uncropped 1920x1080 images at speeds fast enough anyway. It would be nice to crop away areas not of interest to save compute power, or for systems with even more cameras.