otel-profiling-agent icon indicating copy to clipboard operation
otel-profiling-agent copied to clipboard

Backends able to receive OTLP profile signals

Open christophe-kamphaus-jemmic opened this issue 1 year ago • 6 comments

Which backends are able to receive the OTLP profile signals?

I know that the opentelemetry-collector is able to receive it, but where can it forward it for storage and analysis?

I have been able to forward it to Grafana Pyroscope. image Although I have not had much luck with getting Python symbols.

Currently I have a Debian Bookworm application container running many Python services, among other things. This is running on an Azure VM with Docker Enginer. Alongside the application container I have an OTEL Collector (Contrib) container. That OTEL Collector then also exports to the OTLP endpoint for Pyroscope (default port 4040). Pyroscope can be queried from Grafana itself.

JLowe-N avatar Jan 03 '25 23:01 JLowe-N

Does the eBpf profile agent needs to run on the same host as the otel collector which is grpc otlp profile signal server for the agent? How can I configure network interface/ip and port for the agent or which other options I have? I

cforce avatar Jan 17 '25 18:01 cforce

can elasticsearch receive the OTLP profile signal from otel collector ?

hanstf avatar Feb 11 '25 14:02 hanstf

can elasticsearch receive the OTLP profile signal from otel collector ?

Not yet, no. But we are working on profiles support for the elasticsearch collector exporter.

dmathieu avatar Feb 11 '25 15:02 dmathieu

Does the eBpf profile agent needs to run on the same host as the otel collector which is grpc otlp profile signal server for the agent?

opentelemetry-ebpf-profiler needs to run on the host which should be observed, while the opentelemetry-collector / profiling backend able to receive the OTLP signals can run on another host.

How can I configure network interface/ip and port for the agent or which other options I have?

cf. https://github.com/open-telemetry/opentelemetry-ebpf-profiler?tab=readme-ov-file#running This issue also mentions it: https://github.com/open-telemetry/opentelemetry-ebpf-profiler/issues/285

I have been able to forward it to Grafana Pyroscope.

Thanks

Since I posted this issue I have also come across https://github.com/metrico/qryn which claims to be able to ingest Opentelemetry profiling signals and being compatible with Pyroscope API (so usable from Grafana). I haven't tried it yet though.