Support to use "ip" DSL in the requests

[brenocss]

This works:

id: basic-raw-example
info:
  name: Test RAW Template
  author: pdteam
  severity: info

requests:
  - raw:
      - |
        GET / HTTP/1.1
        Host: {{Hostname}}

    extractors:
      - type: dsl
        dsl:
          - ip

But not this:

id: basic-raw-example
info:
  name: Test RAW Template
  author: pdteam
  severity: info

requests:
  - raw:
      - |
        GET / HTTP/1.1
        Host: {{Hostname}}
        Origin: {{ip}}

should be exposed to use in all supported protocols.

[Mzack9999]

@brenocss Unfortunately, the IP information is available only after performing at least one connection to the target. A temporary workaround could be a template like the following one:

id: basic-raw-example
info:
  name: Test RAW Template
  author: pdteam
  severity: info

requests:
  - raw:
      - |
        GET / HTTP/1.1
        Host: {{Hostname}}
        
      - |
        GET / HTTP/1.1
        Host: {{Hostname}}
        Origin: {{extractedip}}

    extractors:
      - type: dsl
        internal: true
        name: extractedip
        dsl:
          - ip

Is there any specific use case you encountered that requires the IP information to be available before the connection?

[brenocss]

Use case

nuclei -t internetdb.yaml  -u google.com
nuclei -t internetdb.yaml  -u https://google.com

It is important that this template can be executed even for entries that are not URLs

requests:
  - raw:
      - |
        @Host: https://internetdb.shodan.io/
        GET /{{ip}} HTTP/1.1
        Host: internetdb.shodan.io

An example of a workaround that would be possible when the cross-protocol data sharing function is working

id: teste

info:
  name: debug
  author: brenocss
  severity: info
  description: Ip 
  reference:
    - https://google.com
  classification:
    cwe-id: CWE-200
  tags: dns,A

dns:
  - name: "{{FQDN}}"
    type: A

    matchers:
      - type: word
        words:
          - "IN\tA"

    extractors:
      - type: regex
        name: extractedip
        group: 1
        # internal: true
        regex:
          - "IN\tA\t(\\S+)"
requests:
  - raw:
      - |
        @Host: https://internetdb.shodan.io/
        GET /{{extractedip}} HTTP/1.1
        Host: internetdb.shodan.io

But that won't work for this, that's why dsl would be a better way to do it

echo hackerone.com | nuclei -t internetdb -u 8.8.8.8

Another problem is to keep redoing unnecessary dns/http requests

[Mzack9999]

Unfortunately, knowing the actual server's IP is impossible before performing at least one connection to the DNS hostname. @ehsandeep What do you think about adding a DNS function to resolve hostname=>ip and assuming that the first available IP is optimistically the one used to perform a successful connection?

[ehsandeep]

@Mzack9999 indeed, it will not only solve this, but also helpful for new helpers at https://github.com/projectdiscovery/nuclei/issues/2341

[brenocss]

Unfortunately, knowing the actual server's IP is impossible before performing at least one connection to the DNS hostname. @ehsandeep What do you think about adding a DNS function to resolve hostname=>ip and assuming that the first available IP is optimistically the one used to perform a successful connection?

Don't we need to preprocess all domains before using for all templates?

it is worth mentioning that when the input is an ip the variable {{ip}} is also not valid

echo https://127.0.0.1 | nuclei -t teste.yaml -duc -debug

[Mzack9999]

@brenocss, the pre-processing phase only lexicographically (fqdn)/range (ip + cidr) deduplicates input items. The DNS resolution, if a domain name is provided, happens within fastdialer at connection time.

On Hold - We need to clarify if the helper function with heuristic assumptions can cover the particular use case

[Mzack9999]

• Blocked by https://github.com/projectdiscovery/nuclei/issues/2341
• Most likely not feasible on first request as the ip info is not available at pre-processing time

[brenocss]

@Mzack9999 don't scan based in ip type ( such as ipv6 or ipv4 ) pre process domains and resolve before running?

[Mzack9999]

@brenocss, that is an excellent point. Still, this happens only in the case of scan-all-ips, as we use a custom ip to connect to.