opentelemetry-collector
opentelemetry-collector copied to clipboard
Extract storage configuration for components into a separate module
Is your feature request related to a problem? Please describe. Components need to use storage extensions for various purposes. There should be a standard way of configuring storage for a component, given the potential for multiple storage extensions coexisting.
See the following contrib issue for examples and some discussion: https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/10915.
Describe the solution you'd like
We should add a configstorage module similar to configauth which defines the configuration, and which other components can then use. For the actual configuration, explicitly setting the storage like it's done here: https://github.com/open-telemetry/opentelemetry-collector/pull/5784 seems to be the consensus. For example:
extensions:
filestorage/example1: {}
filestorage/example2: {}
receivers:
filelog:
storage: filestorage/example1
exporters:
otlphttp:
endpoint: <ENDPOINT>
sending_queue:
enabled: true
storage: filestorage/example2
Additional context The following contrib issue was where the discussion started: https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/10915. Additional discussion in the persistent queue PR: https://github.com/open-telemetry/opentelemetry-collector/pull/5784#discussion_r936009614 Impromptu Collector SIG notes on this topic: https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/12871#issuecomment-1204198890
Components in core which currently do this:
exporterhelper
Components in contrib which currently do this:
filelogreceiversqlqueryreceiverackextensionmongodbatlasreceivergroupbytraceprocessor