💡 FR - Add async processing of CIDRs in discovery job

[thezackm]

multithreaded discovery works great when looking at traditional CIDR ranges, but in cases where a configuration has a list of individual IP addresses, it becomes problematic that each CIDR runs synchronously.

Take for example these 2 configurations:

discovery:
  cidrs:
  - 10.0.0.0/22
...

discovery:
  cidrs:
  - 10.0.0.0/32
  - 10.0.0.1/32
  ...
  - 10.0.3.255/32
...

Both of these configurations contain 1,024 IP addresses, but the first config will have 4 threads running against the single CIDR block, while the 2nd config will run every single entry by itself.

In practice, assuming the default 4 threads and 3sec timeout values, this breaks down as such:

10.0.0.0/22 == 1024 addresses / 4 threads == 256 per thread * 3 sec == 768 seconds max / 60 == 12.8 minutes

vs

1024 addresses * 3 sec == 3072 seconds max / 60 == 51.2 minutes

---

Request that we either expand the threads across multiple CIDRs by default; or we enable parallel processing when we identify that the list is filled with all CIDR values between /29 - /32 (8 to 1 IP address, repesctively)

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.