opentelemetry-collector icon indicating copy to clipboard operation
opentelemetry-collector copied to clipboard

Extract storage configuration for components into a separate module

Open swiatekm opened this issue 3 years ago • 1 comments
trafficstars

Is your feature request related to a problem? Please describe. Components need to use storage extensions for various purposes. There should be a standard way of configuring storage for a component, given the potential for multiple storage extensions coexisting.

See the following contrib issue for examples and some discussion: https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/10915.

Describe the solution you'd like We should add a configstorage module similar to configauth which defines the configuration, and which other components can then use. For the actual configuration, explicitly setting the storage like it's done here: https://github.com/open-telemetry/opentelemetry-collector/pull/5784 seems to be the consensus. For example:

extensions:
  filestorage/example1: {}
  filestorage/example2: {}
receivers:
  filelog:
    storage: filestorage/example1
exporters:
  otlphttp:
    endpoint: <ENDPOINT>
    sending_queue:
      enabled: true
      storage: filestorage/example2

Additional context The following contrib issue was where the discussion started: https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/10915. Additional discussion in the persistent queue PR: https://github.com/open-telemetry/opentelemetry-collector/pull/5784#discussion_r936009614 Impromptu Collector SIG notes on this topic: https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/12871#issuecomment-1204198890

swiatekm avatar Aug 05 '22 17:08 swiatekm

Components in core which currently do this:

  • exporterhelper

Components in contrib which currently do this:

  • filelogreceiver
  • sqlqueryreceiver
  • ackextension
  • mongodbatlasreceiver
  • groupbytraceprocessor

swiatekm avatar Jun 19 '24 09:06 swiatekm