opentelemetry-rust
opentelemetry-rust copied to clipboard
[Bug]: OpenTelemetry log error occurred. cannot send message to batch processor as the channel is closed
What happened?
When Tokio shuts down, I receive a lot of errors about a channel being closed: This happens whether or not I attempt to shutdown (unset) the trace/log providers.
Here is a brief explanation as code:
#[tokio::main]
async fn main() {
// initialise tracing/logging here
opentelemetry::global::shutdown_tracer_provider();
opentelemetry::global::shutdown_logger_provider();
// Without this sleep, I get a different error:
// OpenTelemetry log error occurred. Exporter otlp encountered the following errors: the grpc server returns error (Unknown error): , detailed error message: transport error
tokio::time::sleep(Duration::from_secs(2)).await;
// errors are printed at this point.
}
For a full (hopefully) reproducable version of the code and dependencies, see: https://gist.github.com/NickLarsenNZ/fbbb477230e9992e9a4aa2603edcf54f
I initially raised it in the #otel-rust Slack channel: https://cloud-native.slack.com/archives/C03GDP0H023/p1712590806651679
API Version
0.22.0
SDK Version
0.22.1
What Exporters are you seeing the problem on?
OTLP
Relevant log output
OpenTelemetry log error occurred. cannot send message to batch processor as the channel is closed.
The batch log processor has a background task that exporting the logs. When it's being drop
ed the receiver in the background task will gets dropped too.
After this operation, every attempt to emit logs will result in such error.
I am seeing this too. Is it safe not to call opentelemetry::global::shutdown_tracer_provider();
when the application exits to avoid this error? What are the other implications of doing so?
so if we have this error, it means that the tracer provider has not been shutdown correctly before the application stopped?
I have this error even when I call opentelemetry::global::shutdown_tracer_provider()
before ending my application, is it an expected behavior?
on my side, I realized that this error occurred after cloning the pipeline provider with something like:
let provider = otlp_pipeline
.install_batch(opentelemetry_sdk::runtime::Tokio)
.expect("Failed to build OTLP pipeline")
.provider()
.clone();
and then running opentelemetry::global::shutdown_tracer_provider();
would not shutdown everything (and therefore would gave me the error mentioned above) if the program ends while provider
has not also been dropped