João Duarte

A few items we should scope: 1. the set of metrics we'd like to grab and their description, something like this: | Metric Name | Unit | Description | |--------------------------------------------|--------|----------------------------------------------------------------------|...

> the gradual size estimation vs 1shot One way would be to add an event metadata field to each event and keep updating it as the events is created/modified by...

hi @toby-sutor can you walk me through the setup you had? plain debug logging shouldn't do that: ``` ❯ grep "LogStash::Outputs::ElasticSearch/@passwor" logs -r logs/logstash-plain.log:[2025-04-01T09:30:16,017][DEBUG][logstash.outputs.elasticsearch] config LogStash::Outputs::ElasticSearch/@password = ```

The code has been simplified to assume compressed data and same account id/region. test code was also dramatically reduced by reusing what the collector provides to compare logs, ignoring order...

I believe I have addressed most comments: * a few field renames * removed special handling of certain response elements and request parameters I've also rename the files to adhere...

A few more updates: * cleaned up last renaming references of "cloudtraillogs" in favor of "cloudtraillog", including file renames * updated readme with recent schema changes * reinstated principalId and...

Here are my results: https://gist.github.com/jsvd/4ccbc1776f3bc1679e4751aa0a9500be Key takeaways: 1. in a medium complexity pipeline, 1gb heaps hinder throughput and latency when compared to 4gb heaps; 2. throughput-wise, with 4gb heaps size,...

An example configuration: ``` input { tcp { port => 3333 } } filter { if [x] > 1 { drop {} } } output { stdout {} } ```...

The original idea of having the component + its sources in the same location was taken from the resourcedetectionprocessor, with a few arguments in favor: 1. reduces the spread of...

Expanding on the extension idea, I could foresee the implementation look something like this: ## Lookup processor: Accept a single extension, perform one or more lookups, configure an error mode....