opentelemetry-cpp icon indicating copy to clipboard operation
opentelemetry-cpp copied to clipboard

Segmentation fault during exit in "opentelemetry::v1::context::RuntimeContext::GetCurrent ()" in multithread log export

Open xman1979 opened this issue 1 month ago • 5 comments

Describe your environment linux, ubuntu 22.04, all builds, main branch

Steps to reproduce

Create 2 threads, both are exporting logs in a loop, when program exits, we hit crash with this stack trace:

Thread 4 "struct" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7ffff6741640 (LWP 865935)]
0x000055555590e308 in opentelemetry::v1::context::RuntimeContext::GetCurrent () at /tmp/third-party/opentelemetry-cpp/api/include/opentelemetry/context/runtime_context.h:93
93        static Context GetCurrent() noexcept { return GetRuntimeContextStorage()->GetCurrent(); }
(gdb) bt
#0  0x000055555590e308 in opentelemetry::v1::context::RuntimeContext::GetCurrent ()
    at /tmp/third-party/opentelemetry-cpp/api/include/opentelemetry/context/runtime_context.h:93
#1  0x000055555590d744 in opentelemetry::v1::sdk::logs::Logger::CreateLogRecord (this=0x555556924830)
    at /tmp/third-party/opentelemetry-cpp/sdk/src/logs/logger.cc:70

What is the expected behavior? Should have clear shutdown

What is the actual behavior? Segmentation fault

Additional context I use OTLP http exporter, but I don't think this is the problem, attached the code example here:

  otlp::OtlpHttpLogRecordExporterOptions expOptions;
  expOptions.url = geturl();
  expOptions.content_type = otlp::HttpRequestContentType::kBinary;
  expOptions.ssl_insecure_skip_verify = true;
  auto exporter = otlp::OtlpHttpLogRecordExporterFactory::Create(expOptions);
  auto processor = logs_sdk::SimpleLogRecordProcessorFactory::Create(std::move(exporter));

  std::shared_ptr<logs_sdk::LoggerProvider> loggerProvider(
      logs_sdk::LoggerProviderFactory::Create(std::move(processor)));

  const std::shared_ptr<logs::LoggerProvider> &apiProvider = loggerProvider;
  logs::Provider::SetLoggerProvider(apiProvider);

Both Thread 1 and Thread 2:

  auto loggerProvider = logs::Provider::GetLoggerProvider();
  logger_ = loggerProvider->GetLogger(
        "log",
        "log",
        "1.0.0",
        schema_url);

  for (int i =0; i < 1000; i++) {
    auto sample = logger_->CreateLogRecord();
    logger_->EmitLogRecord(std::move(sample));
    std::this_thread::sleep_for(std::chrono::milliseconds(10));
  }

xman1979 avatar Nov 26 '25 00:11 xman1979

I did some debugging, I believe it is a bug in the lib, attempted a fix here: https://github.com/open-telemetry/opentelemetry-cpp/pull/3767

xman1979 avatar Nov 26 '25 00:11 xman1979

are the threads joined before exit?

Reneg973 avatar Nov 26 '25 05:11 Reneg973

are the threads joined before exit?

yes, something like this

class DataSink {

DataSink() {
  thread_ = std::thread([this] {
    loggingFunc();
  });
}

~DataSink() {
 if (thread_.joinable()) {
    notifyLoggingFunc();
    thread_.join();
  }
}

private:
  std::thread thread_;
}

xman1979 avatar Nov 26 '25 07:11 xman1979

are the threads joined before exit?

yes, something like this

class DataSink {

DataSink() {
  thread_ = std::thread([this] {
    loggingFunc();
  });
}

~DataSink() {
 if (thread_.joinable()) {
    notifyLoggingFunc();
    thread_.join();
  }
}

private:
  std::thread thread_;
}

It just shows the destructor, not the order of destruction and removing modules. I guess during unloading the module

I use OTLP http exporter

gets unloaded before your DataSink is destroyed.

Reneg973 avatar Nov 27 '25 04:11 Reneg973

To investigate in more details.

marcalff avatar Dec 01 '25 21:12 marcalff