Feature: Encrypted traffic metadata

[spendletonliveaction]

Make sure these boxes are signed before submitting your Pull Request -- thank you.

• [x] I have read the contributing guide lines at https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Contributing
• [x] I have signed the Open Information Security Foundation contribution agreement at https://suricata.io/about/contribution-agreement/
• [x] I have updated the user guide (in doc/userguide/) to reflect the changes made (if applicable)

Link to redmine ticket: https://redmine.openinfosecfoundation.org/issues/5372

Describe changes:

This request adds a compile time feature to enable the generation of metadata that can be used to implement encrypted traffic analysis detection as described here: https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/sec_data_eta/configuration/xe-16-6/sec-data-encrypted-traffic-analytics-xe-16-6-book/sec-data-encrypted-traffic-analytics-xe-16-6-book_chapter_01.html

[codecov[bot]]

Codecov Report

Merging #7415 (38a733c) into master (f8bf581) will increase coverage by 0.08%. The diff coverage is 40.74%.

:exclamation: Current head 38a733c differs from pull request most recent head 3875311. Consider uploading reports for the commit 3875311 to get more accurate results

@@            Coverage Diff             @@
##           master    #7415      +/-   ##
==========================================
+ Coverage   75.75%   75.84%   +0.08%     
==========================================
  Files         659      656       -3     
  Lines      185743   189997    +4254     
==========================================
+ Hits       140713   144096    +3383     
- Misses      45030    45901     +871     

Flag	Coverage Δ
fuzzcorpus	60.37% <40.74%> (+0.23%)	:arrow_up:
suricata-verify	51.84% <39.18%> (-0.60%)	:arrow_down:
unittests	61.06% <0.00%> (+0.34%)	:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

[victorjulien]

Looks like cifuzz found an int handling issue:

flow.c:435:53: runtime error: signed integer overflow: 7594870390325015023 * 1000 cannot be represented in type 'long'
13581
    #0 0x613662 in FlowEncryptedTrafficUpdate /src/suricata/src/flow.c:435:53
13582
    #1 0x613662 in FlowHandlePacketUpdate /src/suricata/src/flow.c:581:5
13583
    #2 0x6200f2 in FlowUpdate /src/suricata/src/flow-worker.c:208:5
13584
    #3 0x6200f2 in FlowWorker /src/suricata/src/flow-worker.c:505:17
13585
    #4 0x5b908f in LLVMFuzzerTestOneInput /src/suricata/src/tests/fuzz/fuzz_predefpcap_aware.c:130:13
13586
    #5 0x542d03 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:611:15
13587
    #6 0x5424ea in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool, bool*) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:514:3
13588
    #7 0x544354 in fuzzer::Fuzzer::ReadAndExecuteSeedCorpora(std::__Fuzzer::vector<fuzzer::SizedFile, std::__Fuzzer::allocator<fuzzer::SizedFile> >&) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:826:7
13589
    #8 0x544589 in fuzzer::Fuzzer::Loop(std::__Fuzzer::vector<fuzzer::SizedFile, std::__Fuzzer::allocator<fuzzer::SizedFile> >&) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:857:3
13590
    #9 0x5340bf in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:912:6
13591
    #10 0x55cdd2 in main /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerMain.cpp:20:10
13592
    #11 0x7fa893858082 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x24082)
13593
    #12 0x50d1bd in _start (build-out/fuzz_predefpcap_aware+0x50d1bd)

https://github.com/OISF/suricata/runs/6519648111?check_suite_focus=true

[spendletonliveaction]

Thank you for the feedback. I am going through the issues. I added the redmine ticket here https://redmine.openinfosecfoundation.org/issues/5372

[spendletonliveaction]

Added suricata-verify test for this feature here: https://github.com/OISF/suricata-verify/pull/857

[victorjulien]

Closing due to inactivity. If you're interested in picking this back up, please open a new PR addressing the comments. Thanks!