vector icon indicating copy to clipboard operation
vector copied to clipboard

enhancement(enriching): add optional wildcard search parameter

Open nzxwang opened this issue 7 months ago • 1 comments

Summary

This PR resolves https://github.com/vectordotdev/vector/issues/22920 by enhancing the enrichment functions with support for optional matching against a wildcard as a fallback. It's implementation is a fallback to finding a match against the wildcard in the indexed function to pass the wildcard-matched rows to the follow-up sequential scan which was also updated to include a fallback to matching against the wildcard in row_equals. In the latter, I've refactored the comparison logic to compare_values as I believe its more readable (but I'm happy to be wrong about this). I also tried to update the documentation but I'm happy to correct this as well.

As a side note, I did consider implementing some delimiter parameter which would allow the user to specify delimiter="|" and pass multiple match values per column in the condition field directly but the implementation would be more complicated than simply specifying a wildcard for all fields. Full regex support would be even better but its implementation would be even more complicated.

Change Type

  • [ ] Bug fix
  • [x] New feature
  • [ ] Non-functional (chore, refactoring, docs)
  • [ ] Performance

Is this a breaking change?

  • [ ] Yes
  • [x] No

How did you test this PR?

cargo fmt --all
cargo clippy --workspace --all-targets -- -D warnings
RUST_BACKTRACE=1 cargo vdev test enrichment_tables::file::tests:: 
./scripts/check_changelog_fragments.sh

Does this PR include user facing changes?

  • [x] Yes. Please add a changelog fragment based on our guidelines.
  • [ ] No. A maintainer will apply the "no-changelog" label to this PR.

nzxwang avatar May 19 '25 18:05 nzxwang

Hi @buraizu, thank you for your review and approval. By any chance, are you able to contact one of the vector maintainers to review this PR as well?

nzxwang avatar Jun 11 '25 01:06 nzxwang

Hi @pront , sorry to bug you directly. Would you mind reviewing this PR when you have some time?

nzxwang avatar Jun 24 '25 18:06 nzxwang

Hi @pront thank you for the review. I appreciate your diligence, but would it be okay to merge this without the FindOptions refactoring? In the next few weeks I plan on making more contributions to vector and I will complete it after having this wildcard search functionality available to us.

nzxwang avatar Jul 07 '25 19:07 nzxwang

Sorry for all the noise. I didn't realize that the website docs get parsed into actual tests. This time, I confirmed that cargo vdev test-vrl ran properly before pushing.

nzxwang avatar Jul 08 '25 19:07 nzxwang

Sorry for all the noise. I didn't realize that the website docs get parsed into actual tests. This time, I confirmed that cargo vdev test-vrl ran properly before pushing.

No problem, thank you for the fixes!

pront avatar Jul 09 '25 13:07 pront