open-telemetry
opentelemetry-otlp 0.30.0 panics on export when using reqwest-client
Backtraces
thread 'OpenTelemetry.Metrics.PeriodicReader' panicked at /nix/store/qa5bbmi8ylmbfcq0kbqlysff7g0dhraz-rust-mixed/lib/rustlib/src/rust/library/core/src/ops/function.rs:250:5:
there is no reactor running, must be called from the context of a Tokio 1.x runtime
stack backtrace:
2025-08-12T04:35:31.892787Z DEBUG opentelemetry_sdk: name="BatchSpanProcessor.ExportingDueToTimer"
2025-08-12T04:35:31.893200Z DEBUG opentelemetry-http: name="ReqwestClient.Send"
0: __rustc::rust_begin_unwind
at /rustc/29483883eed69d5fb4db01964cdf2af4d86e9cb2/library/std/src/panicking.rs:697:5
1: core::panicking::panic_fmt
at /rustc/29483883eed69d5fb4db01964cdf2af4d86e9cb2/library/core/src/panicking.rs:75:14
2: core::panicking::panic_display
at /nix/store/qa5bbmi8ylmbfcq0kbqlysff7g0dhraz-rust-mixed/lib/rustlib/src/rust/library/core/src/panicking.rs:268:5
3: tokio::runtime::scheduler::Handle::current::panic_cold_display
at /nix/store/qa5bbmi8ylmbfcq0kbqlysff7g0dhraz-rust-mixed/lib/rustlib/src/rust/library/core/src/panic.rs:100:13
4: tokio::runtime::scheduler::Handle::current
at /home/charles/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/mod.rs:93:27
5: tokio::time::sleep::Sleep::new_timeout
at /home/charles/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/time/sleep.rs:255:22
6: tokio::time::sleep::sleep
at /home/charles/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/time/sleep.rs:127:27
7: core::ops::function::FnOnce::call_once
at /nix/store/qa5bbmi8ylmbfcq0kbqlysff7g0dhraz-rust-mixed/lib/rustlib/src/rust/library/core/src/ops/function.rs:250:5
8: core::option::Option<T>::map
at /nix/store/qa5bbmi8ylmbfcq0kbqlysff7g0dhraz-rust-mixed/lib/rustlib/src/rust/library/core/src/option.rs:1146:29
9: reqwest::async_impl::client::Client::execute_request
at /home/charles/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/reqwest-0.12.22/src/async_impl/client.rs:2515:14
10: reqwest::async_impl::client::Client::execute
at /home/charles/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/reqwest-0.12.22/src/async_impl/client.rs:2443:14
11: opentelemetry_http::reqwest::<impl opentelemetry_http::HttpClient for reqwest::async_impl::client::Client>::send_bytes::{{closure}}
at /home/charles/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/opentelemetry-http-0.30.0/src/lib.rs:89:37
12: <core::pin::Pin<P> as core::future::future::Future>::poll
at /nix/store/qa5bbmi8ylmbfcq0kbqlysff7g0dhraz-rust-mixed/lib/rustlib/src/rust/library/core/src/future/future.rs:124:9
13: opentelemetry_otlp::exporter::http::metrics::<impl opentelemetry_otlp::metric::MetricsClient for opentelemetry_otlp::exporter::http::OtlpHttpClient>::export::{{closure}}
at /home/charles/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/opentelemetry-otlp-0.30.0/src/exporter/http/metrics.rs:39:14
14: <opentelemetry_otlp::metric::MetricExporter as opentelemetry_sdk::metrics::exporter::PushMetricExporter>::export::{{closure}}
at /home/charles/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/opentelemetry-otlp-0.30.0/src/metric.rs:164:78
15: futures_executor::local_pool::block_on::{{closure}}
at /home/charles/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/futures-executor-0.3.31/src/local_pool.rs:316:34
16: futures_executor::local_pool::run_executor::{{closure}}
at /home/charles/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/futures-executor-0.3.31/src/local_pool.rs:90:37
17: std::thread::local::LocalKey<T>::try_with
at /nix/store/qa5bbmi8ylmbfcq0kbqlysff7g0dhraz-rust-mixed/lib/rustlib/src/rust/library/std/src/thread/local.rs:315:12
18: std::thread::local::LocalKey<T>::with
at /nix/store/qa5bbmi8ylmbfcq0kbqlysff7g0dhraz-rust-mixed/lib/rustlib/src/rust/library/std/src/thread/local.rs:279:20
19: futures_executor::local_pool::run_executor
at /home/charles/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/futures-executor-0.3.31/src/local_pool.rs:86:27
20: futures_executor::local_pool::block_on
at /home/charles/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/futures-executor-0.3.31/src/local_pool.rs:316:5
21: opentelemetry_sdk::metrics::periodic_reader::PeriodicReaderInner<E>::collect_and_export
at /home/charles/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/opentelemetry_sdk-0.30.0/src/metrics/periodic_reader.rs:412:9
22: opentelemetry_sdk::metrics::periodic_reader::PeriodicReader<E>::collect_and_export
at /home/charles/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/opentelemetry_sdk-0.30.0/src/metrics/periodic_reader.rs:340:20
23: opentelemetry_sdk::metrics::periodic_reader::PeriodicReader<E>::new::{{closure}}
at /home/charles/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/opentelemetry_sdk-0.30.0/src/metrics/periodic_reader.rs:287:63
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
thread 'OpenTelemetry.Traces.BatchProcessor' panicked at /nix/store/qa5bbmi8ylmbfcq0kbqlysff7g0dhraz-rust-mixed/lib/rustlib/src/rust/library/core/src/ops/function.rs:250:5:
there is no reactor running, must be called from the context of a Tokio 1.x runtime
stack backtrace:
0: __rustc::rust_begin_unwind
at /rustc/29483883eed69d5fb4db01964cdf2af4d86e9cb2/library/std/src/panicking.rs:697:5
1: core::panicking::panic_fmt
at /rustc/29483883eed69d5fb4db01964cdf2af4d86e9cb2/library/core/src/panicking.rs:75:14
2: core::panicking::panic_display
at /nix/store/qa5bbmi8ylmbfcq0kbqlysff7g0dhraz-rust-mixed/lib/rustlib/src/rust/library/core/src/panicking.rs:268:5
3: tokio::runtime::scheduler::Handle::current::panic_cold_display
at /nix/store/qa5bbmi8ylmbfcq0kbqlysff7g0dhraz-rust-mixed/lib/rustlib/src/rust/library/core/src/panic.rs:100:13
4: tokio::runtime::scheduler::Handle::current
at /home/charles/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/runtime/scheduler/mod.rs:93:27
5: tokio::time::sleep::Sleep::new_timeout
at /home/charles/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/time/sleep.rs:255:22
6: tokio::time::sleep::sleep
at /home/charles/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.47.1/src/time/sleep.rs:127:27
7: core::ops::function::FnOnce::call_once
at /nix/store/qa5bbmi8ylmbfcq0kbqlysff7g0dhraz-rust-mixed/lib/rustlib/src/rust/library/core/src/ops/function.rs:250:5
8: core::option::Option<T>::map
at /nix/store/qa5bbmi8ylmbfcq0kbqlysff7g0dhraz-rust-mixed/lib/rustlib/src/rust/library/core/src/option.rs:1146:29
9: reqwest::async_impl::client::Client::execute_request
at /home/charles/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/reqwest-0.12.22/src/async_impl/client.rs:2515:14
10: reqwest::async_impl::client::Client::execute
at /home/charles/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/reqwest-0.12.22/src/async_impl/client.rs:2443:14
11: opentelemetry_http::reqwest::<impl opentelemetry_http::HttpClient for reqwest::async_impl::client::Client>::send_bytes::{{closure}}
at /home/charles/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/opentelemetry-http-0.30.0/src/lib.rs:89:37
12: <core::pin::Pin<P> as core::future::future::Future>::poll
at /nix/store/qa5bbmi8ylmbfcq0kbqlysff7g0dhraz-rust-mixed/lib/rustlib/src/rust/library/core/src/future/future.rs:124:9
13: opentelemetry_otlp::exporter::http::trace::<impl opentelemetry_sdk::trace::export::SpanExporter for opentelemetry_otlp::exporter::http::OtlpHttpClient>::export::{{closure}}
at /home/charles/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/opentelemetry-otlp-0.30.0/src/exporter/http/trace.rs:48:14
14: <opentelemetry_otlp::span::SpanExporter as opentelemetry_sdk::trace::export::SpanExporter>::export::{{closure}}
at /home/charles/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/opentelemetry-otlp-0.30.0/src/span.rs:156:76
15: futures_executor::local_pool::block_on::{{closure}}
at /home/charles/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/futures-executor-0.3.31/src/local_pool.rs:316:34
16: futures_executor::local_pool::run_executor::{{closure}}
at /home/charles/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/futures-executor-0.3.31/src/local_pool.rs:90:37
17: std::thread::local::LocalKey<T>::try_with
at /nix/store/qa5bbmi8ylmbfcq0kbqlysff7g0dhraz-rust-mixed/lib/rustlib/src/rust/library/std/src/thread/local.rs:315:12
18: std::thread::local::LocalKey<T>::with
at /nix/store/qa5bbmi8ylmbfcq0kbqlysff7g0dhraz-rust-mixed/lib/rustlib/src/rust/library/std/src/thread/local.rs:279:20
19: futures_executor::local_pool::run_executor
at /home/charles/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/futures-executor-0.3.31/src/local_pool.rs:86:27
20: futures_executor::local_pool::block_on
at /home/charles/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/futures-executor-0.3.31/src/local_pool.rs:316:5
21: opentelemetry_sdk::trace::span_processor::BatchSpanProcessor::export_batch_sync
at /home/charles/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/opentelemetry_sdk-0.30.0/src/trace/span_processor.rs:494:29
22: opentelemetry_sdk::trace::span_processor::BatchSpanProcessor::get_spans_and_export
at /home/charles/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/opentelemetry_sdk-0.30.0/src/trace/span_processor.rs:472:22
23: opentelemetry_sdk::trace::span_processor::BatchSpanProcessor::new::{{closure}}
at /home/charles/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/opentelemetry_sdk-0.30.0/src/trace/span_processor.rs:397:37
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
This appears to be fixed by enabling the experimental_metrics_periodicreader_with_async_runtime and experimental_metrics_periodicreader_with_async_runtime (I also already had rt-tokio enabled) and then using opentelemetry_sdk::metrics::periodic_reader_with_async_runtime::PeriodicReader and opentelemetry_sdk::trace::span_processor_with_async_runtime::BatchSpanProcessor when creating the SdkMeterProvider and SdkTracerProvider respectively. Those are some long names. Anyway this is probably primarily a documentation bug, if anything.
Possibly related: https://github.com/open-telemetry/opentelemetry-rust/issues/2739#issuecomment-2986058268
reqwest-blocking should be used by default. (reqwest cannot be used with the default periodic_reader). I'll see what doc gaps we have, and address them shortly.
reqwest-blockingshould be used by default.
It is, but I was trying to use the async client because I'm already using #[tokio::main] so it seems more mechanically sympathetic to use the async client. It just wasn't really clear that there were more steps needed than just enabling the reqwest-client feature (and disabling the reqwest-blocking-client feature).
(Also while I'm here, it's unfortunate that the current API design sandwiches non-async functions between async functions. For example, I have to use tokio::task::spawn_blocking or tokio::task::block_in_place in order to call the opentelemetry_sdk::{trace,meter}::{SdkMeterProvider,SdkTracerProvider}::shutdown methods in an async context even though the implementation of shutdown is async internally.)
OTel SDK uses a dedicated own thread for BatchLog/SpanProcessor and for PeriodicReader - inside this thread, it does blocking calls (hence the need for reqwest-blocking).
Yes shutdown can be done the following way: https://github.com/open-telemetry/opentelemetry-rust/blob/main/opentelemetry-otlp/examples/basic-otlp-http/src/main.rs#L171
Or upvote on this issue: https://github.com/open-telemetry/opentelemetry-rust/issues/2476 - this is probably what you are looking for I think...
OTel SDK uses a dedicated own thread for BatchLog/SpanProcessor and for PeriodicReader
Is this also the case for the variants using async clients? I would assume not, right?
Yes shutdown can be done the following way: https://github.com/open-telemetry/opentelemetry-rust/blob/main/opentelemetry-otlp/examples/basic-otlp-http/src/main.rs#L171
Hmm but that blocks the task it's called from doesn't it?
Or upvote on this issue: #2476 - this is probably what you are looking for I think...
Ah, yes, thanks for pointing me to that