Segmentation fault during exit in "opentelemetry::v1::context::RuntimeContext::GetCurrent ()" in multithread log export
Describe your environment linux, ubuntu 22.04, all builds, main branch
Steps to reproduce
Create 2 threads, both are exporting logs in a loop, when program exits, we hit crash with this stack trace:
Thread 4 "struct" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7ffff6741640 (LWP 865935)]
0x000055555590e308 in opentelemetry::v1::context::RuntimeContext::GetCurrent () at /tmp/third-party/opentelemetry-cpp/api/include/opentelemetry/context/runtime_context.h:93
93 static Context GetCurrent() noexcept { return GetRuntimeContextStorage()->GetCurrent(); }
(gdb) bt
#0 0x000055555590e308 in opentelemetry::v1::context::RuntimeContext::GetCurrent ()
at /tmp/third-party/opentelemetry-cpp/api/include/opentelemetry/context/runtime_context.h:93
#1 0x000055555590d744 in opentelemetry::v1::sdk::logs::Logger::CreateLogRecord (this=0x555556924830)
at /tmp/third-party/opentelemetry-cpp/sdk/src/logs/logger.cc:70
What is the expected behavior? Should have clear shutdown
What is the actual behavior? Segmentation fault
Additional context I use OTLP http exporter, but I don't think this is the problem, attached the code example here:
otlp::OtlpHttpLogRecordExporterOptions expOptions;
expOptions.url = geturl();
expOptions.content_type = otlp::HttpRequestContentType::kBinary;
expOptions.ssl_insecure_skip_verify = true;
auto exporter = otlp::OtlpHttpLogRecordExporterFactory::Create(expOptions);
auto processor = logs_sdk::SimpleLogRecordProcessorFactory::Create(std::move(exporter));
std::shared_ptr<logs_sdk::LoggerProvider> loggerProvider(
logs_sdk::LoggerProviderFactory::Create(std::move(processor)));
const std::shared_ptr<logs::LoggerProvider> &apiProvider = loggerProvider;
logs::Provider::SetLoggerProvider(apiProvider);
Both Thread 1 and Thread 2:
auto loggerProvider = logs::Provider::GetLoggerProvider();
logger_ = loggerProvider->GetLogger(
"log",
"log",
"1.0.0",
schema_url);
for (int i =0; i < 1000; i++) {
auto sample = logger_->CreateLogRecord();
logger_->EmitLogRecord(std::move(sample));
std::this_thread::sleep_for(std::chrono::milliseconds(10));
}
I did some debugging, I believe it is a bug in the lib, attempted a fix here: https://github.com/open-telemetry/opentelemetry-cpp/pull/3767
are the threads joined before exit?
are the threads joined before exit?
yes, something like this
class DataSink {
DataSink() {
thread_ = std::thread([this] {
loggingFunc();
});
}
~DataSink() {
if (thread_.joinable()) {
notifyLoggingFunc();
thread_.join();
}
}
private:
std::thread thread_;
}
are the threads joined before exit?
yes, something like this
class DataSink { DataSink() { thread_ = std::thread([this] { loggingFunc(); }); } ~DataSink() { if (thread_.joinable()) { notifyLoggingFunc(); thread_.join(); } } private: std::thread thread_; }
It just shows the destructor, not the order of destruction and removing modules. I guess during unloading the module
I use OTLP http exporter
gets unloaded before your DataSink is destroyed.
To investigate in more details.