suricata icon indicating copy to clipboard operation
suricata copied to clipboard

detect/bytemath: Support out of order options

Open jlucovsky opened this issue 1 year ago • 1 comments

Continuation of #7694

The intent of this PR is to support arbitrary order options for byte_math. During the investigation phase, it was suggested that the parser be converted to Rust for simplification. Thus, the PCRE based parser for byte_math was removed and replaced with a Rust based parser. @jasonish's prototype rules parser work helped guide this PR.

The C unittests were retained and many were added to the Rust based parser.

Issue: 5077

Link to redmine ticket: 5077

Describe changes:

  • Converts the PCRE based parser to Rust.
  • Adds unit tests to the new Rust modules
  • Removes the PCRE parser from detect-bytemath.c
  • Adjusts the C source modules to refer to the Rust definitions

Updates

  • Address review comments from #7694

#suricata-verify-pr: #suricata-verify-repo: #suricata-verify-branch: #suricata-update-pr: #suricata-update-repo: #suricata-update-branch: #libhtp-pr: #libhtp-repo: #libhtp-branch:

jlucovsky avatar Aug 09 '22 12:08 jlucovsky

Codecov Report

Merging #7706 (6f39725) into master (debdff0) will decrease coverage by 0.03%. The diff coverage is 73.07%.

@@            Coverage Diff             @@
##           master    #7706      +/-   ##
==========================================
- Coverage   75.99%   75.95%   -0.04%     
==========================================
  Files         660      660              
  Lines      185705   185532     -173     
==========================================
- Hits       141127   140930     -197     
- Misses      44578    44602      +24     
Flag Coverage Δ
fuzzcorpus 60.65% <78.94%> (-0.04%) :arrow_down:
suricata-verify 52.52% <41.66%> (-0.05%) :arrow_down:
unittests 60.70% <62.50%> (-0.01%) :arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

codecov[bot] avatar Aug 09 '22 18:08 codecov[bot]

WARNING:

field test baseline %
tlpw1_stats_chk
.tcp.rst 131702 105279 125.1%
ips_afp_stats_chk
.flow.end.state.new 12648 10800 117.11%
generic_stats_chk
.capture.kernel_drops 5316770 5654519 94.03%
.tcp.segment_memcap_drop 40284 11729 343.46%
.tcp.reassembly_gap 138888 114099 121.73%
.tcp.insert_data_normal_fail 39214 11358 345.25%

Pipeline 8732 WARNING: THERE IS A KNOWN BAD BASELINE WITH PACKET DROPS. bE MINDFUL OF ANY RESULTS.

suricata-qa avatar Aug 24 '22 19:08 suricata-qa

Continued in #7779

jlucovsky avatar Aug 26 '22 14:08 jlucovsky