vector icon indicating copy to clipboard operation
vector copied to clipboard

enhancement(http_client source): allow VRL in query parameters

Open benjamin-awd opened this issue 9 months ago • 0 comments

Summary

This PR adds support for VRL in query parameters within the http client source.

This is useful for making GET requests with dynamic parameters e.g. a start_time parameter which can help to filter extraneous data (e.g. only get "active" events based on the current timestamp, or events from the last hour)

e.g.

sources:
  http:
    type: http_client
    endpoint: https://endpoint.com
    method: GET
    query:
      timestamp:
        type: vrl
        value: 'to_unix_timestamp(now(), unit: "milliseconds")'

Which will resolve to https://endpoint.com?timestamp=1749291799121 Closes #14621, #22988

Change Type

  • [ ] Bug fix
  • [x] New feature
  • [ ] Non-functional (chore, refactoring, docs)
  • [ ] Performance

Is this a breaking change?

  • [ ] Yes
  • [x] No

How did you test this PR?

Tests have been added, but otherwise we've been running this in a Kubernetes environment with Helm.

Does this PR include user facing changes?

  • [x] Yes. Please add a changelog fragment based on our guidelines.
  • [ ] No. A maintainer will apply the "no-changelog" label to this PR.

Checklist

  • [x] Please read our Vector contributor resources.
    • make check-all is a good command to run locally. This check is defined here. Some of these checks might not be relevant to your PR. For Rust changes, at the very least you should run:
      • cargo fmt --all
      • cargo clippy --workspace --all-targets -- -D warnings
      • cargo nextest run --workspace (alternatively, you can run cargo test --all)

References

benjamin-awd avatar Mar 21 '25 16:03 benjamin-awd