Multiple rectangle of interest

[Honjan]

Please allow multiple rectangle of interest support. Example: dvr-scan -i some_video.mp4 -o some_video_motion_only.avi -roi 50 50 100 100 -roi 200 100 100 150 -roi 100 250 150 50

[Breakthrough]

This should be possible with argparse as per the following StackOverflow post: https://stackoverflow.com/a/51361113

This will require some changes to the scanner itself though, since currently it uses cropping. For multiple ROIs, we need to combine cropping with masking. This should be possible once a solution for #39 has been added.

[Honjan]

Thank you, I look forward to the new version of DVR-Scan.

[ian-oz]

seconded. usage says "[-roi [x0 y0 w h ...]]" which implies it can already do this. I tried many times to do this, including -roi x y w h x y w h and -roi x y w h -roi x y w h (2nd one just clobbers the first). I then looked at the source and realized the misleading usage message is automatically generated and caused by the nargs=* due to -roi also accepting just w h. Unfortunately, as good as this utility is, it is not useful to me without support for multiple roi options.

Adding the ability to mask OUT region(s) would also be very useful, e.g. to deal with a plant flopping in the wind.

Re the overlap with issue #39, I agree the nicest way to handle all these issues would be an arbitrary mask image (polygonal areas are expensive computationally) but as #39 is already quite old, and processing an array of ROIs should be easy to add, could you please consider doing this as a first step.

p.s. the interactive mode via -roi with no arguments is great.

[mmokrejs]

I also hit the need for mutliple and non-horizontal ROIs. @Breakthrough , please clarify the https://dvr-scan.readthedocs.io/en/latest/guide to make it clear only a single, horizontal ROI is supported at the moment.

Tiny glitch:

[DVR-Scan] ROI set: (x,y)/(w,h) = (7,677)/(200,292)

but the cmdline accepts --region-of-interest 7,677 200,292 and manual says the argument is stripped of brackets, slashes and commas but says nothing about spaces. I propose to change the [DVR-Scan] ROI set: line to have exactly the syntax anticipated on the commandline.

[Breakthrough]

@mmokrejs non-rectangular ROIs will be supported in the future using an image mask, that work is tracked under #39.

I'll pull this feature into the next release though, allowing multiple rectangular ROIs. Detection will take place on the smallest sub-set of the frame that covers all ROIs, after which the resulting motion mask will be clipped based on the selected ROIs.

Regarding the command line syntax, I'll look into that - I wasn't sure if some patterns don't work on some OSes or terminals or not, so did not want to prescribe a solution. The documentation is a bit confusing though, I agree - you do need to use at least commas or spaces or brackets to separate the components. I'll see if that wording can be made a bit better.

[mmokrejs]

Thanks for the work on the masks, it is really needed if one needs to re-process old recordings.

How about [DVR-Scan] ROI set: (x,y)/(w,h) = (7,677)/(200,292) , enter '--region-of-interest 7,677 200,292' at the cmdline without the apostrophes

[Breakthrough]

I've made some progress on this and this should be included when DVR-Scan v1.6 is completed. Here's a sample of what's to come:

You can use the mouse to drag existing points or add new ones, to define arbitrary shapes. The GUI won't support multiple shapes yet, but you can specify multiple shapes via command line and config file. The syntax for ROIs will change slightly in v1.6:

• If you specify two points (e.g. --roi 5 5 10 10), it will be interpreted as a rectangle (first point = top left, second point = bottom right)
  • NOTE: The current syntax uses top left + size with -roi
• If you specify three or more points (e.g. --roi 0 0 10 10 5 5) it will be interpreted as a closed polygon
• You can specify --roi multiple times to add multiple regions (they can be overlapping or disjoint)

Example usage (two overlapping rectangles for simplicity):

dvr-scan -i video.mp4 --roi 0 0 50 100 --roi 0 0 100 0 100 50 0 50

This would limit detection to the area of the box (0,0) to (50,100) and (0,0) to (100,50) (i.e. an L-shape in the top-left corner).

This isn't functional yet, but should be soon - I'll post an update when something is available for folks to test out. In the develop branch, you can use multiple horizontal ROIs currently, but I'm working on extending the model to use polygons as per the above screenshot. I'm also working out how to make this work properly with config files, and to add proper backwards compatibility with the existing ROI specification (using a point + size rather than top left/bottom right). Thanks!

P.S. For now I won't pursue using PNG images as masks and instead focus on polygons, but if anyone requires that they are free to file a new issue.

[Breakthrough]

This is now complete and is included in the latest release (v1.6). Feedback is most welcome!

[mmokrejs]

Thank you @Breakthrough for the changes, finally I am getting to test them in v1.6. Some newbie comments:

I thought dvr-scan -i foo.mp4 -r -s foo.roi will ensure the coords from the region editor will get saved there. No, they did not.

The region editor should show the coords at the bottom of the page so that poor users could at least take a pic of the [age an retype it. Don't lough.

One user realizes there is Print points the following gets printed on STDERR:

[DVR-Scan] Region data for CLI:
--region 1218 1316 1596 1434 1954 1594 2058 1728 2012 1872 1360 1666 828 1528 770 1326

But hey, this syntax is not recognized and users are instructed to use commas as a separator. But other parts of the docs show for example -a 50 50 100 50 75 75 so is it really necessary to use the commas?

dvr-scan --min-event-length 2 --time-before-event 0.1s --time-post-event 0.1s --threshold 0.1 --kernel-size -1 --bg-subtractor MOG2 --output-mode opencv -i foo.mp4 --region-of-interest 1218 1316 1596 1434 1954 1594 2058 1728 2012 1872 1360 1666 828 1528 770 1326  -o foo.mkv
[DVR-Scan] DVR-Scan 1.6
[DVR-Scan] Error: Invalid value for ROI: 1218 1316 1596 1434 1954 1594 2058 1728 2012 1872 1360 1666 828 1528 770 1326. ROI must be specified as a rectangle of the form `x,y,w,h` or the max window size `w,h` (commas/spaces are ignored). For example: -roi 200,250 50,100

Seems the --region is not recognized anymore but one is ought to use --region-of-interest instead.

Seems the testcase https://github.com/Breakthrough/DVR-Scan/commit/b0673512e84d8ce3e94e5c8235cd97c23ca73fcc#diff-4e8715c7a425ee52e74b7df4d34efd32e8c92f3e60bd51bc2e1ad5943b82032e uses only a rectangular region. Please make a testcase with 5 or more points.

I kept changing the camera setup so ideally I want to define for each input file a separate .roi file so the region editor should write it for me with minimum button clicks. If the whole is that I did not type S while within the region editor than maybe call it if user decides to quit the window?

[DVR-Scan] Region editor active. Press h to show controls.
[DVR-Scan] ROI Window Controls:

Editor:
  Mask On/Off         Key: M
  Start Scan          Key: Space, Enter
  Quit                Key: Escape
  Save                Key: S
  Load                Key: O
  Undo                Key: Z
  Redo                Key: Y
  Print Points        Key: C

Regions:
  Add Point           Key: A,  Mouse: Left
  Delete Point        Key: X,  Mouse: Middle
  Add Region          Key: Shift + T
  Delete Region       Key: Shift + G
  Select Region       Key: 1 - 9
  Next Region         Key: L
  Previous Region     Key: K

Display:
  Downscale +/-       Key: W(+), E (-)
  Antialiasing        Key: Q
  Window Mode         Key: R

Something maybe does not work ideally in my setup on Arch Linux due to:

qt.qpa.plugin: Could not find the Qt platform plugin "wayland" in "/home/foo/.local/share/pipx/venvs/dvr-scan/lib/python3.11/site-packages/cv2/qt/plugins"
[DVR-Scan] Region editor active. Press h to show controls.

The commit b067351 could have probably referenced this issue #58 to make it easier to find the new features.

Finally, after I hit the S in ROI I received a window asking me where to save the coords. But why does not it respect dvr-scan -i foo.mp4 -o foo.mkv -r -s some.roi at first?

OK, so I saved it under $HOME/some.txt and moved it under my ${long_path}/foo.roi. But how how to load it then? --region-of-interest does not accept file path, neither --roi does. In the end, what attracted my eyes was -R, at least if there was ROI.txt instead of FILE.txt ...

usage: dvr-scan [-h] [-V] [-L] -i video_file [video_file ...] [-d path] [-o video.avi] [-m mode] [-so] [-c settings.cfg] [-r] [-a [X0 Y0 X1 Y1 X2 Y2 ...]] [-R FILE.txt]
                [-s FILE.txt] [-b type] [-t value] [-k size] [-l time] [-tb time] [-tp time] [-st time] [-dt time] [-et time] [-bb [smooth_time]] [-tc] [-fm]
                [-mo motion_mask.avi] [-df factor] [-fs num_frames] [-q] [--logfile file] [-v type]

[Breakthrough]

Thanks for the feedback @mmokrejs. I'll look more into your comments soon, but just want to type up a quick reply.

I thought dvr-scan -i foo.mp4 -r -s foo.roi will ensure the coords from the region editor will get saved there. No, they did not.

I assume this happened because you aborted the scan or closed the window. However I agree the documentation there is misleading. I will see if I can add a pop-up window with a warning when you close the window with unsaved changes. The intention was only for the file to be saved if the scan was started, but I agree it should warn you and give the option to save changes if you do not.

[DVR-Scan] Region data for CLI:
--region 1218 1316 1596 1434 1954 1594 2058 1728 2012 1872 1360 1666 828 1528 770 1326

Regions should be added using --add-region, so the output there is a bug - I will make a note to fix that. See the updated docs for an explanation of the new flags. This is why the old -roi flag only works with rectangles. Only --add-region is compatible with polygons.

I kept changing the camera setup so ideally I want to define for each input file a separate .roi file so the region editor should write it for me with minimum button clicks. If the whole is that I did not type S while within the region editor than maybe call it if user decides to quit the window?

As before I agree and will look into adding this, thank you for pointing this out.

In the end, what attracted my eyes was -R, at least if there was ROI.txt instead of FILE.txt ...

This is a good point, I can also update that to make it more clear.

More comments to come, but thank you for the feedback on the new features!

[mmokrejs]

Would be great to improve the info messages to report how many rectangular regions will be used and how many polygons will be used. Currently there is:

[DVR-Scan] Limiting detection to 1 region.
[DVR-Scan] Using subtractor MOG2 with kernel_size = 7 (auto)
[DVR-Scan] Scanning input video for motion events...

If region is rectangular the a new variable should be used to report number of polygons setup in the above output.

I would also propose to rename --add-region to --add-polygon to make it clear what it does.

I ended up with 1218 1316 1596 1434 1954 1594 2058 1728 2012 1872 1360 1666 828 1528 770 1326 in the .roi file so I hope it was parsed as a polygon. The bounding box surrounding detected scenes was shown rectangular, though. Could the polygon used for detection be shown as well when -bb is specified at the cmdline so one could verify what region(s)/polygon(s) were actually used?

Thank you for the cleanup.

The region editor picks the first frame. Provided my videos start always at the midnight the images are difficult to be used to delineate motion regions. Some button to skip by next say 10% of the file would be very helpful. By that one could jump into a day-light scene after a while.

[mmokrejs]

Only --add-region is compatible with polygons.

dvr-scan: error: argument -a/--add-region: Regions can only contain numbers and the following characters: , / ( )
  Input: /foo/bar.roi
$
$ cat /foo/bar.roi
0 930 334 802 410 614 494 442 460 346 432 242 510 60 806 62 906 174 860 212 906 258 1016 290 988 328 1032 392 1088 520 1286 516 1396 376 2560 178 2560 642 1648 762 1452 718 1356 862 1540 1270 1966 1406 2560 1310 2560 1920 0 1920
$

I see, I need

values=`cat $foo.roi`
cmdline="dvr-scan ... --add-region $values ..."`

[Breakthrough]

Thanks again for the feedback @mmokrejs, I've fixed the incorrect output so now it shows you the full command when you hit C in the region editor, and changed the CLI option to appear as -R REGIONS.txt. I also filed #161 to track the issue where closing the window does not save changes.

Will make sure these get in for the next release (v1.6.1).