opentelemetry-collector
opentelemetry-collector copied to clipboard
open-telemetry
Implement pipeline instrumentation
The "Pipeline Component Telemetry" RFC defines generic metrics and logs to monitor the data passing through the interface between pipeline components, as well as a mechanism for capturing this telemetry. It now needs to be implemented, probably as a wrapper around the Consumer interface inserted at the pipeline graph level.
This new telemetry should include the component-identifying attributes defined in the same RFC. Instead of rederiving these attributes, it should make use of the implementation of #12217.
cc @djaglowski
If I'm not mistaken there isn't already an issue tracking this right?
I don't think this can be considered completed until the "outcome = refused" case has been implemented. (To be clear: I do think it makes sense to put that in a separate PR since the implementation may be a bit subtle.)
Just saw this issue and wanted to ask if this work will make metrics like otelcol_exporter_send_failed_<signal> being deprecated. Thanks!
@iblancasa I don't think there's any concrete plans to do that yet, although it's something to think about considering the overlap in functionality.
I think the receiverhelper and processorhelper metrics could definitely be deprecated once we have confidence the pipeline metrics are usable (enabled by default, most users have support for instrumentation scope attributes, etc.). The exporterhelper metrics will still have value, since the pipeline metrics considers "successfully enqueued" to be outcome = success, even if the export operation fails later; but maybe we could remodel those metrics to look more like the pipeline metrics for consistency, as if the queue was its own (sub)component?
@iblancasa I don't think there's any concrete plans to do that yet, although it's something to think about considering the overlap in functionality.
I see. And totally makes sense. It should be something to consider at some point.
I think the receiverhelper and processorhelper metrics could definitely be deprecated once we have confidence the pipeline metrics are usable (enabled by default, most users have support for instrumentation scope attributes, etc.). The exporterhelper metrics will still have value, since the pipeline metrics considers "successfully enqueued" to be
outcome = success, even if the export operation fails later; but maybe we could remodel those metrics to look more like the pipeline metrics for consistency, as if the queue was its own (sub)component?
Very likely will be features that could not be observed from the pipelines (I guess). But yes, it can be a great idea to implement them in this way.