open-telemetry
[Feature]: OTLP Exporter Retry and Persistence
Related Problems?
No response
Describe the solution you'd like:
Per the spec for OTLP Exporters, "Transient errors MUST be handled with a retry strategy. This retry strategy MUST implement an exponential back-off with jitter to avoid overwhelming the destination until the network is restored or the destination has recovered."
Similar to the equivalent feature for opentelemetry-dotnet, the Rust OpenTelemetry implementation must support retry of failed requests to be conformant to the OTLP Exporter spec. The retry mechanism should also be able to leverage an optional persistent store for additional resilience to unexpected process restarts. Implementation should follow a similar pattern to the dotnet implementation.
One set of PRs should implement retry logic for the OTLP Exporter per the OpenTelemetry spec.
Another set of PRs should introduce persistent blob abstractions (i.e. similar to OpenTelemetry.PersistentStorage.Abstractions and a file-based implementation (i.e. similar to OpenTelemetry.PersistentStorage.FileSystem). This set of PRs should also include updates to the retry logic to allow opt-in use persistent storage for retry.
References
- OTLP Specification
- OTLP Exporter Specification
- [opentelemetry-dotnet] OTLP Exporter Retry and Persist Feature
- [opentelemetry-dotnet] Add persistent storage to exporter
Considered Alternatives
No response
Additional Context
No response
I don't believe the spec requires the second part of this:
introduce persistent blob abstractions (i.e. similar to OpenTelemetry.PersistentStorage.Abstractions and a file-based implementation (i.e. similar to OpenTelemetry.PersistentStorage.FileSystem). This set of PRs should also include updates to the retry logic to allow opt-in use persistent storage for retry.
There is a long standing spec change proposal open-telemetry/opentelemetry-specification/issues/3645. I suggest that this not required for the sake of marking OTLP compliant and ultimately closing #3061 . @cijothomas wdyt?
Correct. Persistence storage is not a blocker for stable release. Its a nice-to-have capability, and is not spec-ed yet.
