startbootstrap icon indicating copy to clipboard operation
startbootstrap copied to clipboard

Update cloudtrail parser

Open alexgoedeke opened this issue 3 years ago • 6 comments

One line description of pull request

add current log format for aws cloudtrail parser.

Description:

Related issue (if applicable): fixes #4182

Notes:

All contributions to Plaso undergo code review. This makes sure that the code has appropriate test coverage and conforms to the Plaso style guide.

One of the maintainers will examine your code, and may request changes. Check off the items below in order, and then a maintainer will review your code.

Checklist:

  • [x] Automated checks (Travis, Codecov, Codefactor )pass
  • [x] No new new dependencies are required or l2tdevtools has been updated
  • [ ] Reviewer assigned

alexgoedeke avatar Aug 09 '22 19:08 alexgoedeke

Codecov Report

Base: 85.66% // Head: 85.67% // Increases project coverage by +0.01% :tada:

Coverage data is based on head (74bccff) compared to base (4356f3d). Patch coverage: 87.64% of modified lines in pull request are covered.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4187      +/-   ##
==========================================
+ Coverage   85.66%   85.67%   +0.01%     
==========================================
  Files         404      405       +1     
  Lines       34656    34745      +89     
==========================================
+ Hits        29688    29768      +80     
- Misses       4968     4977       +9     
Impacted Files Coverage Δ
plaso/parsers/aws_cloudtrail_log.py 87.64% <87.64%> (ø)
plaso/multi_process/extraction_engine.py 77.64% <0.00%> (+0.38%) :arrow_up:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

codecov[bot] avatar Aug 09 '22 22:08 codecov[bot]

@alexgoedeke Thanks for the contribution! At first glance, it seems like the test data you are using here differs from the test data in the original parser (e.g. not in JSONL format) and more closely matches what is provided in https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-log-file-examples.html - looks like the format is in fact JSON and not JSONL. @jonathan-greig

jleaniz avatar Aug 09 '22 22:08 jleaniz

Hi Alexander, thanks for this PR but Juan is correct that the Cloudtrail log parser is designed for logs saved in JSON-L format and your changes seem to change the parser to use JSON. dftimewolf with the aws_logging_collect can be used to save cloudtrail logs in JSON-L format.

jonathan-greig avatar Sep 02 '22 03:09 jonathan-greig

@alexgoedeke I think it would make sense to have this parser in addition to the existing one, rather than as a replacement. This new parser would seem to work on log data directly exported from AWS Cloudtrail. Would you be willing to adjust this PR accordingly?

jleaniz avatar Sep 02 '22 03:09 jleaniz

@jleaniz Yes, this makes sense. I adjusted the PR accordingly.

alexgoedeke avatar Sep 05 '22 08:09 alexgoedeke

@alexgoedeke how is this parser different from https://github.com/log2timeline/plaso/blob/main/plaso/parsers/jsonl_plugins/aws_cloudtrail_log.py ?

any example logs that can indicate the difference between the JSON and JSONL variants?

joachimmetz avatar Sep 10 '22 13:09 joachimmetz

@alexgoedeke what is the status of this? Should I close this PR for now and reopen when you have the time work on it?

joachimmetz avatar Nov 01 '22 05:11 joachimmetz

per previous comment closing PR for now

joachimmetz avatar Mar 26 '23 08:03 joachimmetz