beats icon indicating copy to clipboard operation
beats copied to clipboard

[9.1](backport #48020) feat: add `add_cloud_metadata` processor to OTel Beat processor

Open mergify[bot] opened this issue 2 weeks ago • 3 comments

Proposed commit message

Add add_cloud_metadata processor to the OTel Beat processor.

Checklist

  • [x] My code follows the style guidelines of this project
  • [x] I have commented my code, particularly in hard-to-understand areas
  • [x] I have made corresponding changes to the documentation
  • ~~[ ] I have made corresponding change to the default configuration files~~
  • [x] I have added tests that prove my fix is effective or that my feature works. Where relevant, I have used the stresstest.sh script to run them under stress conditions and race detector to verify their stability.
  • ~~[ ] I have added an entry in ./changelog/fragments using the changelog tool.~~

How to test this PR locally

  1. Include the processor in Elastic Agent by adding it to https://github.com/elastic/elastic-agent/blob/89e0e9a5b0ebb376773de813a1fd2c0b2f30e74a/internal/pkg/otel/components.go.
  2. Build the Elastic Agent binary
  3. Run the Agent on a cloud instance, e.g. an AWS EC2 instance, with an OTel config that includes the Beat processor.

Example OTel config:

service:
  pipelines:
    logs:
      receivers:
        - filebeatreceiver
      processors:
        - beat
receivers:
  filebeatreceiver:
    filebeat:
      inputs:
        - type: filestream
          id: logs-from-files
          paths:
            - /var/log/*.log 
    processors: []
    output:
      otelconsumer:
    path.data: /tmp/1210/data
    path.logs: /tmp/1210/logs
    queue.mem:
      flush.timeout: 0
processors:
  beat:
    processors:
      - add_cloud_metadata:
exporters:
  debug:
    sending_queue:
      enabled: false
    use_internal_logger: false
    verbosity: detailed
```<hr>This is an automatic backport of pull request #48020 done by [Mergify](https://mergify.com).

mergify[bot] avatar Dec 11 '25 14:12 mergify[bot]

Cherry-pick of e1630bed1fa73f3caaecf8945e2990c3158112bb has failed:

On branch mergify/bp/9.1/pr-48020
Your branch is up to date with 'origin/9.1'.

You are currently cherry-picking commit e1630bed1.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	modified:   x-pack/otel/processor/beatprocessor/README.md
	modified:   x-pack/otel/processor/beatprocessor/processor.go
	modified:   x-pack/otel/processor/beatprocessor/processor_test.go

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   x-pack/filebeat/tests/integration/otel_test.go

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

mergify[bot] avatar Dec 11 '25 14:12 mergify[bot]

:robot: GitHub comments

Just comment with:

  • run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

github-actions[bot] avatar Dec 11 '25 14:12 github-actions[bot]

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

elasticmachine avatar Dec 11 '25 14:12 elasticmachine

This pull request has not been merged yet. Could you please review and merge it @andrzej-stencel? 🙏

mergify[bot] avatar Dec 15 '25 05:12 mergify[bot]