kyma
kyma copied to clipboard
[Telemetry Operator] Make collecting Kubernetes annotations and labels optional
Description
Kubernetes labels and annotations may add a significant overhead to the size of a log record. For instance through the last applied configuration as an annotation. Collecting these metadata should be optional and to be configured per LogPipeline.
Acceptance Criteria
- Collecting Kubernetes labels and annotations should be configurable per LogPipeline in the output settings
- Annotations are dropped by default. Labels are usually meaningful as you would search for logs via labels.
Removing a nested key by using the record_motifier filter does not seem to work: https://github.com/fluent/fluent-bit/issues/2152. Using a Lua filter seem to be a workaround.