WIP: Add autoexport to automatically configure trace exporters

[MikeGoldsmith]

This PR adds an autoexport package similar to autoprop that can detect and configure trace exporters, or use provided ones if none are set in environment variables.

This is a WIP and can only detect and configure an OTLP gRPC exporter - other exporters can be added later.

Note: common exporters (eg otlp, jaeger, etc) cannot be pre-populated into the registry during init because they would not take into account environment variables configured before autoexport is called. This is valuable, as it both supports testing and when combined with the proposed generic config launcher (see below).

• #2592
• https://github.com/honeycombio/opentelemetry-go-contrib/pull/400

Known limitations:

• Only one exporter per type can be configured, eg a single "otlp" exporter can be configured
• Does not consider metrics exporters yet because they are slightly more complicated because metrics exporters are wrapped by a controller (push / pull)

[codecov[bot]]

Codecov Report

Merging #2753 (d98ccee) into main (e9956ff) will increase coverage by 0.0%. The diff coverage is 80.4%.

Additional details and impacted files

@@          Coverage Diff          @@
##            main   #2753   +/-   ##
=====================================
  Coverage   79.3%   79.3%           
=====================================
  Files        163     165    +2     
  Lines      10325   10422   +97     
=====================================
+ Hits        8193    8274   +81     
- Misses      2000    2012   +12     
- Partials     132     136    +4     

Impacted Files	Coverage Δ
exporters/autoexport/exporter.go	77.5% <77.5%> (ø)
exporters/autoexport/registry.go	82.4% <82.4%> (ø)

... and 1 file with indirect coverage changes

[MikeGoldsmith]

Hey @MrAlias - could you give this another pass for feedback please? Thanks

[jmacd]

This seems fundamentally different than the autoprop package in the sense that it supports registration of span Exporters (a part of a provider), whereas the thing configured in autoprop is a whole provider of propagation functionality. When I consider how to auto-configure a Span exporter, that includes the notion of auto-configuring the processor at the same time. Some span exporters will want to use the batch processor, and some will not--although these are independent interfaces I do not believe they should be auto-configured separately.

So, the thing I expect a registry for auto-configured-span-exporters to return is a TracerProviderOption which encapsulates the processor and exporter (i.e., a whole pipeline). Likewise for auto-configuring metrics exporteres, I expect a registry for auto-configured-metric-exporters to configure MetricReader instances appropriately for some exporter and return a MeterProviderOption.

[MrAlias]

This seems fundamentally different than the autoprop package in the sense that it supports registration of span Exporters (a part of a provider), whereas the thing configured in autoprop is a whole provider of propagation functionality. When I consider how to auto-configure a Span exporter, that includes the notion of auto-configuring the processor at the same time. Some span exporters will want to use the batch processor, and some will not--although these are independent interfaces I do not believe they should be auto-configured separately.

So, the thing I expect a registry for auto-configured-span-exporters to return is a TracerProviderOption which encapsulates the processor and exporter (i.e., a whole pipeline). Likewise for auto-configuring metrics exporteres, I expect a registry for auto-configured-metric-exporters to configure MetricReader instances appropriately for some exporter and return a MeterProviderOption.

This sounds out of scope of what this package is intended to accomplish. Rather it is a concern of the https://github.com/open-telemetry/opentelemetry-go-contrib/pull/2592 proposal.

Similar to how autoprop does not provide automatic registration of the returned value, this package is a building block that a complete SDK is intended to be built from.

[jmacd]

I'm just feeling wary of building automatic-configurators for individual building blocks of the SDK on the assumption that they are always perfectly orthogonal from one another. I would prefer instead of configuring Samplers, Processors, and Exporters independently (for spans) for us to configure feature-level units of code which sometimes means configuring more than one SDK interface with a coordinated implementation. Here is an example of a combined java-contrib Sampler and Processor implementation.

[MikeGoldsmith]

Thanks for the feedback all, I'll be following up on queries and addressing feedback over the next couple of days.

[MikeGoldsmith]

I've gone through and addressed all feedback.

Main changes are:

• Used Options pattern for providing fallback exporter
• Updated registry to populate with OTLP exporter during init() and simplified registry retrieval logic
• Added autoexport to dependabot and versions yamls
  • @MrAlias I added to experimental-instrumenations as autoprop was there, let me know if you'd prefer somewhere else

Thanks all 😄

[MikeGoldsmith]

Hmm - I've had another thought regarding using this package with the proposed go contrib launcher, and am wondering if moving the otlp exporter creation to init is a good idea. It means a vendor who wants to modify settings, eg override the exporter's endpoint, cannot do it before its created.

I think we need to delay creating the default (otlp) exporter until it's requested by calling SpanExporter().

[MikeGoldsmith]

Hmm - I've had another thought regarding using this package with the proposed go contrib launcher, and am wondering if moving the otlp exporter creation to init is a good idea. It means a vendor who wants to modify settings, eg override the exporter's endpoint, cannot do it before its created.

I think we need to delay creating the default (otlp) exporter until it's requested by calling SpanExporter().

Thinking more on this, in the scenario someone wants to alter the default OTLP exporter programatically (eg headers, endpoint, etc) - would we expect them to register a new exporter? Many users of the autoexport package will likely want to alter some settings.

For the launcher when using the OTLP exporter, if we want to edit the endpoint, we'd then have to create a new exporter anyway so makes no sense for using the autoexport package at all.

[MikeGoldsmith]

I've talked myself round in a circle. I think what we have is good. If an app, or an intermediate layer such as the launcher, wants to provide an alternative exporter configuration, then you should build and register an exporter 👍🏻

[MikeGoldsmith]

@open-telemetry/go-approvers @open-telemetry/go-maintainers looking for another round of feedback on this PR please 👍🏻

[alexrudd]

Hey, are there any updates on this? Would be a really great feature :)

[MikeGoldsmith]

I’ll update to resolve conflicts. Agree it would be good to get in, hopefully there is more capacity now metrics SDK changes are complete.

[MikeGoldsmith]

@MrAlias @dashpole @pdelewski I've updated to use a SpanExporter factory instead of maintaining SpanExporter instances. Let me know what you think.

[pdelewski]

LGTM

[MikeGoldsmith]

@MrAlias @pellared I've addressed your feedback. Please re-review 👍🏻

[MikeGoldsmith]

@pellared How's this?

[james-callahan]

Is there a no-op exporter we can use with autoexport?

[pellared]

Is there a no-op exporter we can use with autoexport?

I created https://github.com/open-telemetry/opentelemetry-go-contrib/issues/4013