feat(sources): add access to URL path in custom VRL auth

[byronwolfman]

trafficstars

Summary

This adds access to .path, in addition to .address and .headers in VRL scripts for custom auth strategy. This is useful for tying specific authorization headers to specific paths e.g. for Vector aggregators receiving payloads from different vendors on different URL paths, each with distinct authorization strategies.

Change Type

• [ ] Bug fix
• [x] New feature
• [ ] Non-functional (chore, refactoring, docs)
• [ ] Performance

Is this a breaking change?

• [ ] Yes
• [x] No

How did you test this PR?

Added unit tests and also built a basic Vector pipeline:

sources:
  in:
    type: "http_server"
    address: "0.0.0.0:8080"
    strict_path: false
    auth:
      strategy: "custom"
      source: |-
        if starts_with(string!(.path), "/vendor1/") && .headers.authorization == "test" {
          true
        } else if starts_with(string!(.path), "/vendor2/") && .headers."x-some-auth" == "abc123" {
          true
        } else {
          false
        }

sinks:
  console:
    inputs: ["in"]
    target: "stdout"
    type: "console"
    encoding:
      codec: "json"

Auth successes:

# Matching path and header
$ curl -X POST localhost:8080/vendor1/ -H "authorization: test"

# Matching path and header
$ curl -X POST localhost:8080/vendor2/ -H "x-some-auth: abc123"

Auth failures:

# No header or path
$ curl -X POST localhost:8080
{"code":401,"message":"Auth failed"}

# Header but no path
$ curl -X POST localhost:8080/ -H "authorization: test"
{"code":401,"message":"Auth failed"}

# Header but no path
$ curl -X POST localhost:8080/ -H "x-some-auth: abc123"
{"code":401,"message":"Auth failed"}

# Mismatched headers/paths
$ curl -X POST localhost:8080/vendor1/ -H "x-some-auth: abc123"
{"code":401,"message":"Auth failed"}

$ curl -X POST localhost:8080/vendor2/ -H "authorization: test"
{"code":401,"message":"Auth failed"}

Does this PR include user facing changes?

• [x] Yes. Please add a changelog fragment based on our guidelines.
• [ ] No. A maintainer will apply the "no-changelog" label to this PR.

Notes

• Please read our Vector contributor resources.
• Do not hesitate to use @vectordotdev/vector to reach out to us regarding this PR.
• The CI checks run only after we manually approve them.
  • We recommend adding a pre-push hook, please see this template.
  • Alternatively, we recommend running the following locally before pushing to the remote branch:
    • cargo fmt --all
    • cargo clippy --workspace --all-targets -- -D warnings
    • cargo nextest run --workspace (alternatively, you can run cargo test --all)
    • ./scripts/check_changelog_fragments.sh
• After a review is requested, please avoid force pushes to help us review incrementally.
  • Feel free to push as many commits as you want. They will be squashed into one before merging.
  • For example, you can run git merge origin master and git push.
• If this PR introduces changes Vector dependencies (modifies Cargo.lock), please run cargo vdev build licenses to regenerate the license inventory and commit the changes (if any). More details here.

References

Closes: https://github.com/vectordotdev/vector/issues/23163

All good/sensible changes were cargo-culted from https://github.com/vectordotdev/vector/pull/22850. Any errors are of my own making.

[bits-bot]

All committers have signed the CLA.

[byronwolfman]

• Merged upstream/main into branch (now up-to-date)
• Specified author in changelog per validate-changelog CI job

The validate-changelog CI job is still complaining about an unrelated changelog entry which is not part of this PR, however:

invalid fragment contents: author option was specified but fragment elasticsearch-encoding.fix.md contains no authors.

[pront]

• Merged upstream/main into branch (now up-to-date)
• Specified author in changelog per validate-changelog CI job

The validate-changelog CI job is still complaining about an unrelated changelog entry which is not part of this PR, however:

invalid fragment contents: author option was specified but fragment elasticsearch-encoding.fix.md contains no authors.

Regarding the failing changelog CI check, we made some recent changes, cc @thomasqueirozb to check this

[thomasqueirozb]

• Merged upstream/main into branch (now up-to-date)
• Specified author in changelog per validate-changelog CI job

The validate-changelog CI job is still complaining about an unrelated changelog entry which is not part of this PR, however:

invalid fragment contents: author option was specified but fragment elasticsearch-encoding.fix.md contains no authors.

For the changelog CI we made some recent, cc @thomasqueirozb to check this

Fixed 🙂