Cluster ports numbers in host list

[chr-b]

Please describe your feature request:

Port numbers provided in the host input and the template file should be merged. The resulting list should be made unique.

Describe the use case of this feature:

My use case is a template that checks both the user provided input port and a default port. Sometimes these ports might be identical, sometimes they might be different. As a template author I cannot know the host list that the Nuclei user provides as input.

Example multiple-ports.yaml:

id: multiple-ports

info:
  name: multiple-ports
  author: me
  severity: info

network:
  - host:
      - "{{Host}}:{{Port}}"
      - "{{Host}}:3306"

Now, when I call this with nuclei -v -t multiple-ports.yaml -u localhost:3306, the output is as follows:

[INF] Using Nuclei Engine 2.7.7 (latest)
...
[INF] Templates loaded for scan: 1
[VER] Sent TCP request to localhost:3306
[VER] Sent TCP request to localhost:3306

Observed behaviour: two requests are sent to port number 3306. Two findings will be reported when a matcher is added. Expected behaviour: Nuclei processes the host list and eliminates duplicate entries. In that case, only one request should be made and only one result/finding reported.

This can probably be generalized to also include http://, etc. prefixes in the duplicate elimination.