uNetworking
Double-Free of UserData when gracefully closing a Websocket from Server-Side
Hi, i am trying to implement a graceful server Shutdown. Either i am doing something wrong, or there is a fundamental bug in the implementation. ASAN complains (rightfully) about a double-free attempt.
My Shutdown-Code involves:
loop->defer([this]
{
for (auto* con: websocketConnections)
con->end(1000, "Server shutting down");
if (appPtr)
appPtr->close();
});
The code of con->end() currently finishes with the following lines:
/* Emit close event */
if (webSocketContextData->closeHandler) {
webSocketContextData->closeHandler(this, code, message);
}
((USERDATA *) this->getUserData())->~USERDATA();
As we can see, the destructor of UserData is called at the very end.
However webSocketContextData->closeHandler() already brings me into uWS::TemplatedApp::ws in App.hpp, line 263 - 439.
Within this function there is this snippet:
/* Copy all handlers */
webSocketContext->getExt()->openHandler = std::move(behavior.open);
webSocketContext->getExt()->messageHandler = std::move(behavior.message);
webSocketContext->getExt()->drainHandler = std::move(behavior.drain);
webSocketContext->getExt()->subscriptionHandler = std::move(behavior.subscription);
webSocketContext->getExt()->closeHandler = std::move([closeHandler = std::move(behavior.close)](WebSocket<SSL, true, UserData> *ws, int code, std::string_view message) mutable {
if (closeHandler) {
closeHandler(ws, code, message);
}
/* Destruct user data after returning from close handler */
((UserData *) ws->getUserData())->~UserData(); // <<<<<<<<<<<<<<<<<<<<<<<<<-----
});
webSocketContext->getExt()->pingHandler = std::move(behavior.ping);
webSocketContext->getExt()->pongHandler = std::move(behavior.pong);
Where i have marked the destructor-call.
So, i am running into a code-path where the destructor is definitely called twice.
This is the full call-stack
[nelson] <lambda#1>::operator()(uWS::WebSocket<true, true, WebsocketData> *, int, std::string_view) App.h:383 <----------------- is calling the destructor
[nelson] std::__invoke_impl<void, <lambda#1> &, uWS::WebSocket<true, true, WebsocketData> *, int, std::string_view>(std::__invoke_other, <lambda#1> &, uWS::WebSocket<true, true, WebsocketData> *&&, int &&, std::string_view &&) invoke.h:63
[nelson] std::__invoke_r<void, <lambda#1> &, uWS::WebSocket<true, true, WebsocketData> *, int, std::string_view>(<lambda#1> &, uWS::WebSocket<true, true, WebsocketData> *&&, int &&, std::string_view &&) invoke.h:113
[nelson] std::move_only_function<void (uWS::WebSocket<true, true, WebsocketData> *, int, std::string_view)>::_S_invoke<<lambda#1> >(std::_Mofunc_base *, uWS::WebSocket<true, true, WebsocketData> *, int, std::string_view &&) mofunc_impl.h:219
[nelson] std::move_only_function<void (uWS::WebSocket<true, true, WebsocketData> *, int, std::string_view)>::operator() mofunc_impl.h:184
[nelson] uWS::WebSocket<true, true, WebsocketData>::end WebSocket.h:266 <----------------- will call destructor again once this callstack finishes.
[nelson] <lambda#1>::operator()() const server.cpp:234
[nelson] std::__invoke_impl<void, <lambda#1> &>(std::__invoke_other, <lambda#1> &) invoke.h:63
[nelson] std::__invoke_r<void, <lambda#1> &>(<lambda#1> &) invoke.h:113
[nelson] std::move_only_function<void ()>::_S_invoke<<lambda#1> >(std::_Mofunc_base *) mofunc_impl.h:219
[nelson] std::move_only_function<void ()>::operator() mofunc_impl.h:184
[nelson] uWS::Loop::wakeupCb Loop.h:50
[nelson] us_loop_run epoll_kqueue.c:147
[nelson] uWS::Loop::run Loop.h:216
[nelson] uWS::run Loop.h:233
[nelson] uWS::TemplatedApp<true>::run App.h:571
[nelson] ServerImpl::runApp server.cpp:215
[nelson] std::__invoke_impl<void, void (ServerImpl::*)(std::string), ServerImpl *, std::string> invoke.h:76
[nelson] std::__invoke<void (ServerImpl::*)(std::string), ServerImpl *, std::string> invoke.h:98
[nelson] std::thread::_Invoker<std::tuple<void (ServerImpl::*)(std::string), ServerImpl *, std::string> >::_M_invoke<0ul, 1ul, 2ul> std_thread.h:303
[nelson] std::thread::_Invoker<std::tuple<void (ServerImpl::*)(std::string), ServerImpl *, std::string> >::operator() std_thread.h:310
[nelson] std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (ServerImpl::*)(std::string), ServerImpl *, std::string> > >::_M_run std_thread.h:255
[libstdc++.so.6] std::execute_native_thread_routine 0x00007ffff64e51a4
[libasan.so.8] asan_thread_start 0x00007ffff785e29b
[libc.so.6] <unknown> 0x00007ffff62a57eb
[libc.so.6] <unknown> 0x00007ffff632918c
I setup the Websocket-functionality with
// WebSocket handlers
app.ws<WebsocketData>("/:0", {
.upgrade = std::bind_front(&ServerImpl::upgradeWebsocket, this),
.open = std::bind_front(&ServerImpl::onWebsocketOpen, this),
.message = [](auto *ws, std::string_view msg, uWS::OpCode op)
{
ws->send(msg, op); // Echo
},
.close = std::bind_front(&ServerImpl::onWebsocketClose, this)
});
With WebsocketData containing a std::string which is attempted to be destructed twice.
Note to self:
Could be WebSocket->end followed by WebSocket->close that doesn't work.
I don't remember the details but I think App.close will call WebSocket::close on all websockets. So the bug can probably be reproduced with any call to WebSocket::end followed immediately by WebSocket::close
You really don't mix end and close. Either you end, or you close. Not both. So it's probably missing a check in .close.
Just doing appPtr->close(); crashes ASAN too.
Just doing con->end(...) is also enough to trigger the bug.
In fact, my application never reaches appPtr->close() with ASAN enabled because con->end() kills it via ASAN.
So, its not about the mixing.
I just wanted to provide a nice close-code and a reason. That is why I wanted to close the connections manually.
I feel like con->end(code, reason) is fundamentally broken as soon as WebsocketData has non-trivial members.
So far I commented out the destructor-call at the end of end(). I rather deal with a leak than a double free.
Connect to it via wscat -c ws://localhost:1234 or a similar tool. As soon as you connect it will defer a close operation with code and reason. While doing so, it will destruct WebsocketData twice. Either you compile with ASAN and crash directly, or the destructor of WebsocketData triggers the __builint_trap()
The crash occurs with #if 1 or #if 0. It does not matter.
//
// Created by albi on 13.06.25.
//
#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Wdeprecated-declarations"
#pragma GCC diagnostic ignored "-Wshadow"
#pragma GCC diagnostic ignored "-Wnrvo"
#include "App.h"
#pragma GCC diagnostic pop
#include <sanitizer/asan_interface.h>
#include <string>
#include <iostream>
int instanceCount = 0;
struct WebsocketData
{
std::string string;
volatile bool destructed = false;
WebsocketData():
string("default-constructed-"+std::to_string(instanceCount++))
{
std::cout << "WebsocketData " << string << std::endl;
}
WebsocketData(const WebsocketData& other):
string("Copy of \"" + other.string+"\"")
{
std::cout << "WebsocketData " << string << std::endl;
}
WebsocketData(WebsocketData&& other) noexcept:
string("Moved from \"" + other.string + "\"")
{
std::cout << "WebsocketData " << string << std::endl;
other.string = "empty/moved from";
}
~WebsocketData()
{
std::cout << "WebsocketData destructed: " << string << std::endl;
if (destructed)
__builtin_trap();
destructed = true;
}
WebsocketData& operator=(const WebsocketData& other) = delete;
WebsocketData& operator=(WebsocketData&& other) = delete;
};
uWS::WebSocket<false, true, WebsocketData>* connection = nullptr;
uWS::App* appPtr = nullptr;
int main(const int argc, const char *argv[])
{
#ifdef __SANITIZE_ADDRESS__
__asan_set_error_report_callback([](const char*) { __builtin_trap(); });
#endif
uWS::App app{};
appPtr = &app;
app.ws<WebsocketData>("/*",
{
.open = [](uWS::WebSocket<false, true, WebsocketData> *ws)
{
std::cout << "Client connected" << std::endl;
connection = ws;
uWS::Loop::get()->defer([]
{
#if 1
if (connection)
connection->end(1000, "Bye Bye");
#else
if (appPtr)
appPtr->close();
#endif
});
},
.close = [](uWS::WebSocket<false, true, WebsocketData> *ws, int, std::string_view)
{
std::cout << "Client disconnected" << std::endl;
connection = nullptr;
}
}).listen(1234, [](auto *listenSocket)
{
if (listenSocket)
std::cout << "Server listening on port 1234" << std::endl;
else
std::cerr << "Failed to listen on port 1234" << std::endl;
}).run();
appPtr = nullptr;
return 0;
}