detection-rules icon indicating copy to clipboard operation
detection-rules copied to clipboard

[enhancement] In esql validation, allow any order of metadata

Open frederikb96 opened this issue 8 months ago • 4 comments

Pull Request

Issue link(s): #4575

Summary - What I changed

Very simple fix to allow any order of esql metadata by adjusting the regex in the validation check.

When using the metadata in the order referenced in the documentation we get a validation error for ESQL with an error message which makes it difficult to find the problem.

Better is to simply allow this official order of metadata :)

How To Test

Checklist

  • [x] Added a label for the type of pr: bug, enhancement, schema, maintenance, Rule: New, Rule: Deprecation, Rule: Tuning, Hunt: New, or Hunt: Tuning so guidelines can be generated
  • [x] Added the meta:rapid-merge label if planning to merge within 24 hours
  • [x] Secret and sensitive material has been managed correctly
  • [x] Automated testing was updated or added to match the most common scenarios
  • [x] Documentation and comments were added for features that require explanation

Contributor checklist

frederikb96 avatar Mar 28 '25 18:03 frederikb96

  1. I also realized that the check is not validating multiple index patterns correctly, also added the regex for this.
  2. Added support for having newline between "|" and the required "keep" which is often used to structure esql rules in GUI

:slightly_smiling_face:

frederikb96 avatar Mar 31 '25 11:03 frederikb96

I used flake to get correct line length and updated code.

@Mikaayenson Anything else that needs to be done, or can this be merged? :)

frederikb96 avatar Apr 09 '25 14:04 frederikb96

@Mikaayenson @eric-forte-elastic

Just small ping, if this can be merged too :)

frederikb96 avatar Apr 23 '25 07:04 frederikb96

@Mikaayenson @eric-forte-elastic

Anything which is blocking here? :)

frederikb96 avatar May 26 '25 10:05 frederikb96

@Mikaayenson @eric-forte-elastic

I rebased the branch, cleaned up the commit history.

Would highly appreciate it if you can merge this PR and bump version if needed etc. It is just a tiny adjustment to not throw errors when a rule contains a permutation of keywords or newlines, that's all :)

frederikb96 avatar Jul 03 '25 15:07 frederikb96

@frederikb96 we're thinking about handling esql validation via the stack. I'll keep you posted. We may not need this.

Mikaayenson avatar Jul 16 '25 19:07 Mikaayenson

@frederikb96 we've decided that it would be helpful to merge in your changes, in addition to other checks for ESQL queries.

The last this left is testing. Would you mind incorporating this patch with the unit test into your pull request?

test_schemas.patch

traut avatar Jul 26 '25 10:07 traut

Closing in favor of https://github.com/elastic/detection-rules/pull/4956 to complete unit tests

Mikaayenson avatar Aug 01 '25 21:08 Mikaayenson