opentelemetry-java
opentelemetry-java copied to clipboard
open-telemetry
How to modify log data in the log pipeline?
There's AutoConfigurationCustomizer.addLogExporterCustomizer which gives a nice hook to modify log data, but it occurs after the BatchLogProcessor, and so doesn't have access to the Context.
LogProcessors run before the BatchLogProcessors, but are not able to modify log data (which makes them unlike SpanProcessors, but also makes them much simpler from concurrency/performance perspective).
Short of making LogProcessors similar to SpanProcessors and allowing them to modify the logs, wdyt of adding AutoConfigurationCustomizer.addBatchLogProcessorCustomizer to allow modifying log data before it goes to the BatchLogProcessor which you still have access to the Context?
An additional problem is that log processors only have access to LogData - there are no methods to mutate. This is problematic because it means that if trying to implement a log processor that extracts data from context, even if you provide your own LogData implementation (i.e. a DelegatingLogData), downstream log processors won't see any of the changes. To accomplish this you'd have to have a log processor that delegates to the batch processor.
I made this example to get a better feel for what it would take to do context extraction via an autoconfigure extension point. Yuck.
discussed briefly with @anuraaga on Tue and he was in favor of changing the LogProcessors to mimic SpanProcessors (i.e. introduce ReadWriteLog similar to ReadWriteSpan)
a good example use case for (mutable) LogProcessor is stamping baggage on logs, e.g. https://github.com/open-telemetry/opentelemetry-java-instrumentation/issues/1391
IMO baggage isn't an ideal example since it's supposed to be handled by OTel itself.
I'm still for having log processor to mimic tracing but if a use case is needed to justify it, it's not baggage ;)
IMO baggage isn't an ideal example since it's supposed to be handled by OTel itself.
Could you expand on this? I think you're saying there should be built in mechanisms to extract baggage onto logs, but I'm not aware of such mechanisms on spans and metrics. Metrics do have experimental support via a baggage attribute processor for views, but it's not part of the spec.
but I'm not aware of such mechanisms
Yeah that's why "supposed to be handled" :P The point is mainly that it would be possible to stamp baggage even without making Log mutable, by the SDK doing it directly before emitting the record.