otel-profiling-agent icon indicating copy to clipboard operation
otel-profiling-agent copied to clipboard

Is there any otel collector version that supports this new feature?

Open Lincyaw opened this issue 10 months ago • 11 comments

I tried to run my own otel collector to collect the data, however, it shows that some of the new features of signal is not implemented.

time="2024-04-17T20:41:17.626067005Z" level=error msg="Failed to handle mapping for PID 1700136, file /usr/bin/clickhouse: failed to load deltas: failed UpdateExeIDToStackDeltas for FileID e71a2d0df314abcd: no map available for 3256297 stack deltas"
time="2024-04-17T20:41:18.285213757Z" level=error msg="Failed to handle mapping for PID 1700137, file /usr/bin/clickhouse: failed to load deltas: failed UpdateExeIDToStackDeltas for FileID e71a2d0df314abcd: no map available for 3256297 stack deltas"
time="2024-04-17T20:41:23.997951426Z" level=info msg="Attached tracer program"
time="2024-04-17T20:41:24.050963144Z" level=info msg="Attached sched monitor"
time="2024-04-17T20:41:24.052807028Z" level=info msg="Environment variable KUBERNETES_SERVICE_HOST not set"
time="2024-04-17T20:41:24.632531045Z" level=error msg="Request failed: rpc error: code = Unimplemented desc = unknown service opentelemetry.proto.collector.profiles.v1.ProfilesService"
time="2024-04-17T20:41:25.882740002Z" level=error msg="Failed to handle mapping for PID 1700137, file /usr/bin/clickhouse: failed to load deltas: failed UpdateExeIDToStackDeltas for FileID e71a2d0df314abcd: no map available for 3256297 stack deltas"
time="2024-04-17T20:41:29.133657342Z" level=error msg="Request failed: rpc error: code = Unimplemented desc = unknown service opentelemetry.proto.collector.profiles.v1.ProfilesService"
time="2024-04-17T20:41:33.961619601Z" level=error msg="Request failed: rpc error: code = Unimplemented desc = unknown service opentelemetry.proto.collector.profiles.v1.ProfilesService"
time="2024-04-17T20:41:43.844658570Z" level=error msg="Request failed: rpc error: code = Unimplemented desc = unknown service opentelemetry.proto.collector.profiles.v1.ProfilesService"
time="2024-04-17T20:41:49.753826484Z" level=error msg="Request failed: rpc error: code = Unimplemented desc = unknown service opentelemetry.proto.collector.profiles.v1.ProfilesService"
^Ctime="2024-04-17T20:41:51.601494731Z" level=info msg="Stop processing ..."
time="2024-04-17T20:41:51.601518358Z" level=info msg="Exiting ..."

Lincyaw avatar Apr 17 '24 20:04 Lincyaw

It seems that I need to modify the makefile of the otel-collector, and use the corresponding version of otel-proto, since this feature is not merged into the otel-proto yet, thus I can not simply change the version of otel-protocol in the makefile of otel-collector.

https://github.com/open-telemetry/opentelemetry-collector/blob/main/Makefile#L185

image

Lincyaw avatar Apr 17 '24 21:04 Lincyaw

https://github.com/petethepig/opentelemetry-collector/pull/1

I guess this version probably can work, can the authors please verify it? or it there any way that can allow me to export the data with the current otel-collector?

Lincyaw avatar Apr 17 '24 21:04 Lincyaw

Hi @Lincyaw :wave:

As you have mentioned, https://github.com/open-telemetry/opentelemetry-proto/pull/534 is not merged yet and to my knowledge, there are no OTel collectors implementing this protocol yet.

At the moment, I can not tell if https://github.com/petethepig/opentelemetry-collector/pull/1 is based exactly on https://github.com/open-telemetry/opentelemetry-proto/pull/534 or if a previous version is implemented. So we did not test this agent with https://github.com/petethepig/opentelemetry-collector/pull/1. We hope, that https://github.com/open-telemetry/opentelemetry-proto/pull/534 gets merged soon and OTel collector and/or other implementations will use it.

For the mean time we provide devfiler to visualize data.

florianl avatar Apr 18 '24 05:04 florianl

@Lincyaw You can print the logs to stdout, so you won't need to set up a remote RPC server. Does that meet your needs?

cfc4n avatar Apr 18 '24 13:04 cfc4n

We hope, that https://github.com/open->telemetry/opentelemetry-proto/pull/534 gets >merged soon and OTel collector and/or other >implementations will use it.

It‘s merged meanwhile

cforce avatar Jun 22 '24 17:06 cforce

We hope, that https://github.com/open->telemetry/opentelemetry-proto/pull/534 gets >merged soon and OTel collector and/or other >implementations will use it.

It‘s merged meanwhile

I see. The tag is experimental, so it also needs to change the otel-profiling-agent's implementation to adopt the experimental tag, otherwise when sending the messages, it will warn that the proto is not matched.

Lincyaw avatar Jun 23 '24 01:06 Lincyaw

Is there a issue for this (assigned) already?

cforce avatar Jun 23 '24 13:06 cforce

Not yet I think. I am also waiting/watching for the solution.

Lincyaw avatar Jun 23 '24 14:06 Lincyaw

There is ongoing work, around https://github.com/open-telemetry/opentelemetry-collector/pull/10375, to bring the new OTel profiling signal into the OTel collector.

florianl avatar Jun 24 '24 05:06 florianl

I wouldn't expect the collector to support this before the end of the summer though.

dmathieu avatar Jun 24 '24 07:06 dmathieu

@Lincyaw Did you manage to get around the issue in the meanwhile somehow?

ShimiTaNaka avatar Aug 12 '24 17:08 ShimiTaNaka