Extending YAML to support include and placeholders preprocessing

[Mzack9999]

Proposed changes

This PR implements the YAML include directive (# !include:path/to/file.yaml). The files can be either .yaml (static blocks) or .yamlc (template blocks, with mixed yaml for payload generation)

Checklist

• [x] Pull request is created against the dev branch
• [ ] All checks passed (lint, unit/integration/regression tests etc.) with my changes
• [ ] I have added tests that prove my fix is effective or that my feature works
• [ ] I have added necessary documentation (if appropriate)

Examples

requests.yaml

- |
  GET /1 HTTP/1.1
  Host: {{Hostname}}
  Accept: */*

- |
  GET /2 HTTP/1.1
  Host: {{Hostname}}
  Accept: */*

matcher.yamlc

# var1:
#   - access[_-]?token
#   - account[_-]?sid
#   - admin[_-]?email
#   - adzerk[_-]?api[_-]?key
#   - algolia[_-]?admin[_-]?key[_-]?1
matchers:
  - type: word
    words:
      {{range .var1}}- common_prefix{{ . }}common_suffix
      {{end}}

template.yaml

id: CVE-2014-3206

info:
  name: test
  author: test
  severity: info
  description: test
  tags: info

requests:
  - raw:
    # !include:requests.yaml
    unsafe: true
    # !include:matcher.yamlc

Run with:

echo http://192.168.1.1 | go run . -t template.yaml -v -vv -debug

[Mzack9999]

Description

To improve and ease the integration with https://github.com/projectdiscovery/nuclei-burp-plugin and web-based YAML standard marshaler/unmarshaler, it would be preferred to replace the existing pattern placeholder:

# !include:requests.yaml

with the YAML marshalable:

import: requests.yaml

So that the information is not lost during conversion operations. This implies that only YAML consistent blocks would be supported (for example, it will be not feasible to include external files within a multiline string)

Todo

• [ ] Replace (or add support) # !include:requests.yaml with import: requests.yaml