opentelemetry-rust
opentelemetry-rust copied to clipboard
OTel Logs with no heap allocation
Opening a parent issue to track making it possible to do OTel Logging with zero heap allocation. This requires support from
- Logging library (
tracing
is used as example, which already has zero allocation and delegates most work to layer/subscriber) - OTel Bridge API - this delegates most work to OTel Logging SDK and has no heap allocation now.
- OTel Log SDK - Currently, this is where all heap allocation occurs!
- Exporter - Exporting to etw, user_events can be synchronously, and hence can be fully done without heap allocation.
Making OTel Log SDK heap allocation free requires addressing the below at the minimum:
- https://github.com/open-telemetry/opentelemetry-rust/issues/1919 - Avoid vec! for passing
LogRecord
to Exporter. - https://github.com/open-telemetry/opentelemetry-rust/issues/1920 - Avoid allocating vec! for storing upto "N" attributes
- https://github.com/open-telemetry/opentelemetry-rust/issues/1921 - Avoid String allocation. For
tracing
mostly
For sync exporters
The overall flow would be (using tracing
as example)
tracing -> otel-tracing-layer -> create LogRecord
on stack, populate it using callback/visitor from tracing-subscriber
. -> pass mut ref of LogRecord
to exporting processor -> exporter ->serialize and write to transport (etw/user-events/lttng)
The above can be achieved fully on stack. This issue is targeting the above only.
For async exporters For exporters requiring Batching for efficiency (eg: OTLP over gRPC, HTTP), some Ring buffer or Channels would be needed. There would be clone/copy costs for those scenarios, but it maybe possible to avoid heap allocation entirely. This will be tracked separately.
Also need some tooling to ensure the assumptions are correct. i.e measure the actual heap allocation and confirm it is indeed 0! Above is based on manual code inspection only.