opentelemetry-collector-contrib icon indicating copy to clipboard operation
opentelemetry-collector-contrib copied to clipboard

Published 20 hours ago • verified publisher icon open-telemetry

[extension/awslogsencodingextension] Support CloudTrail log events

Open jsvd opened this issue 6 months ago • 2 comments

Description

This changeset follows the same structure as vpc flow log, s3 access log and subscription filter logs but now dedicated to CloudTrail log events.

Link to tracking issues

This is part of the effort described in https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/38627:

It would be possible to configure the extension with the specific format, such as CloudWatch log group subscription filter, CloudTrail, etc. By having them co-located, we can avoid proliferation of extensions, and better ensure consistency across the formats (e.g. set cloud.* SemConv fields consistently).

Testing

The log samples were taken from https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-log-file-examples.html#cloudtrail-log-file-examples-section.

This unmarshaller is tested against startInstances and createUser events.

Documentation

Updated documentation to reflect new CloudTrail Logs unmarshalling.

jsvd avatar May 23 '25 14:05 jsvd

CLA Signed

The committers listed above are authorized under a signed CLA.

  • :white_check_mark: login: jsvd / name: João Duarte (1d2f0dc3a60bfa72bee7493651b181aaaf6de866, 614a055f60246e8fd8980aff7361b1cc416578ec, 38868c94f73b6acc061356b29b7500017e46ce13, c3ee0847c284e442fd082eec51ef84f32ccb43f7, 525fbaefce22044faddca96ba0a704371c8a5bfc, d7a0ec8573802ced597477e59acd53bb3236e5ed, f031b83dd248c1c54ad7f3d1cb6eca7d6a035203, 03a628c085d866ed16a6c6a07c966e8a2ed6542e, f33b8ff39b294c8bae0d828e566113ddca2b5aba, 5cc22ac85cb3bceb61a98213156305bc6e829f58, 489427b674b8144a5c7689bfd4850eb8f6afe881, 7da6532fc0453139a65f903af546d0812b0255e4, 944dd127b12a0dd9bfcca082d0a5d8dd96f94375, 43ce5c94224fa99e6e6276fbbed3aa491d704487, 5c232d3dad130a0c55880704a96e1182ab96b8e9, 686a38d61fd0d394d039ed0a4d276ac3c57b26d8, dd38cf8e4c38b2050b215f3278997e2c2b6ebe22, 5fb8bb04b2a6d921826775967639e359fad0c1e1, 955e1b0c4dff0d60f16d8d39348cfcc80a70f895, d5679b8fad2f793fdd657b7e9d64049601ad6cde, 6b73b0e5c3b6d8797e90188097a05f9343616c64, 8522eb348147bc86a8e704a9e51d31e6302ffc37, af7261bf54773912aa0abd5699d3bb470626d4b1)
  • :white_check_mark: login: edmocosta / name: Edmo Vamerlatti Costa (e4281146db9c4d3afbea2e46b3016bc4fabfb815)

linux-foundation-easycla[bot] avatar May 23 '25 14:05 linux-foundation-easycla[bot]

The code has been simplified to assume compressed data and same account id/region. test code was also dramatically reduced by reusing what the collector provides to compare logs, ignoring order of records.

jsvd avatar May 28 '25 10:05 jsvd

I believe I have addressed most comments:

  • a few field renames
  • removed special handling of certain response elements and request parameters I've also rename the files to adhere to the rest of the unmarshalers.

jsvd avatar Jun 20 '25 10:06 jsvd

A few more updates:

  • cleaned up last renaming references of "cloudtraillogs" in favor of "cloudtraillog", including file renames
  • updated readme with recent schema changes
  • reinstated principalId and userName, populate with whatever the event brings leaving redaction to the processing stage.
  • rebased against master due to the gzip fix

jsvd avatar Jun 27 '25 09:06 jsvd