Crow
Crow copied to clipboard
CrowCpp
Memory leak when websocket sends data
When I sent data by calling the send_data() function, I constructed a std:: string temporary object, but a memory leak occurred. The data held by the temporary object does not seem to be released. Can someone help me ?
crow::websocket::connect* conn;
char* data_ptr;
std::size_t data_size;
/* accept connect .... */
data_ptr=new char[6220800];
data_size=6220800;
conn->send_binary(std::string(data_ptr,data_size)); //memory leak!!!
delete [] data_ptr;
// but temporary string memory is not release
Use gcc12 compiler and C++20 on openSUSE system
Is there a reason for new char etc and not using a std::string or std::unique_ptr<char[]> for the data (raii)? Also, I'm not fully convinced sending ~6MB of data in a single blob over WS is a good idea :D Might depend on the app. Regarding the leak: you sure it's the std::string leaking it? Did you confirm this using valgrind or similar? WS is the thing I use the most from crow. I do not leak any memory with them. I also use gcc12 with C++20, so it's a similar environment.
I used GNU Glibc to check for memory leakage issues and analyzed it in conjunction with gdb. I found that the std::string object created memory of size 0x55c1a, but it has not been released since then. After reviewing the source code of the send_binary function, I found that it uses move semantics internally. After all, the ownership of the temporary string object was handed over to crow after the internal memory was created. I think it may be that crow did not release this memory, but I am not familiar with ASIO and WB programming
@MichaelSB Here is mtrace log
@ /usr/lib64/libstdc++.so.6:(_Znwm+0x1c)[0x7f6cec83653c] + 0x3d95ec9f0 0x5eec1a <---- Not released afterwards
@ ./RemoteServer:(_ZN4asio11aligned_newEmm+0x78)[0x7053af] + 0x7f6c8bb55460 0x70
@ /usr/lib64/libstdc++.so.6:(_ZNSt7__cxx1118basic_stringstreamIcSt11char_traitsIcESaIcEED1Ev+0x54)[0x7f6cec8c3784] - 0x3c27e5ac0
@ ./RemoteServer:(_ZN4asio14aligned_deleteEPv+0x18)[0x70541c] - 0x7f6c8bb55460
@ ./RemoteServer:(_ZN4asio14aligned_deleteEPv+0x18)[0x70541c] - 0x7f6cb4002e90
@ ./RemoteServer:(_ZN4asio14aligned_deleteEPv+0x18)[0x70541c] - 0x7f6ce3b54ec0
@ ./RemoteServer:(_ZN4asio11aligned_newEmm+0x78)[0x7053af] + 0x7f6cb4002e90 0x200
@ /usr/lib64/libstdc++.so.6:(_Znwm+0x1c)[0x7f6cec83653c] + 0x7f6ce0000b60 0x201
@ /usr/lib64/libstdc++.so.6:(_Znwm+0x1c)[0x7f6cec83653c] + 0x7f6ce0000d70 0x401
@ /usr/lib64/libstdc++.so.6:(_ZNSt7__cxx1115basic_stringbufIcSt11char_traitsIcESaIcEE8overflowEi+0x161)[0x7f6cec8c4821] - 0x7f6ce0000b60
@ /usr/lib64/libstdc++.so.6:(_Znwm+0x1c)[0x7f6cec83653c] + 0x7f6ce3b54ec0 0x801
@ /usr/lib64/libstdc++.so.6:(_ZNSt7__cxx1115basic_stringbufIcSt11char_traitsIcESaIcEE8overflowEi+0x161)[0x7f6cec8c4821] - 0x7f6ce0000d70
@ /usr/lib64/libstdc++.so.6:(_Znwm+0x1c)[0x7f6cec83653c] + 0x7f6ce3b556d0 0x1001
@ /usr/lib64/libstdc++.so.6:(_ZNSt7__cxx1115basic_stringbufIcSt11char_traitsIcESaIcEE8overflowEi+0x161)[0x7f6cec8c4821] - 0x7f6ce3b54ec0
@ /usr/lib64/libstdc++.so.6:(_Znwm+0x1c)[0x7f6cec83653c] + 0x7f6ce3b566e0 0x2001
@ /usr/lib64/libstdc++.so.6:(_ZNSt7__cxx1115basic_stringbufIcSt11char_traitsIcESaIcEE8overflowEi+0x161)[0x7f6cec8c4821] - 0x7f6ce3b556d0
@ /usr/lib64/libstdc++.so.6:(_Znwm+0x1c)[0x7f6cec83653c] + 0x7f6ce3b586f0 0x4001
@ /usr/lib64/libstdc++.so.6:(_ZNSt7__cxx1115basic_stringbufIcSt11char_traitsIcESaIcEE8overflowEi+0x161)[0x7f6cec8c4821] - 0x7f6ce3b566e0
@ /usr/lib64/libstdc++.so.6:(_Znwm+0x1c)[0x7f6cec83653c] + 0x7f6ce3b5c700 0x8001
@ /usr/lib64/libstdc++.so.6:(_ZNSt7__cxx1115basic_stringbufIcSt11char_traitsIcESaIcEE8overflowEi+0x161)[0x7f6cec8c4821] - 0x7f6ce3b586f0
@ /usr/lib64/libstdc++.so.6:(_Znwm+0x1c)[0x7f6cec83653c] + 0x7f6ce3b64710 0x10001
@ /usr/lib64/libstdc++.so.6:(_ZNSt7__cxx1115basic_stringbufIcSt11char_traitsIcESaIcEE8overflowEi+0x161)[0x7f6cec8c4821] - 0x7f6ce3b5c700
@ /usr/lib64/libstdc++.so.6:(_Znwm+0x1c)[0x7f6cec83653c] + 0x7f6ce3b74720 0x20001
@ /usr/lib64/libstdc++.so.6:(_ZNSt7__cxx1115basic_stringbufIcSt11char_traitsIcESaIcEE8overflowEi+0x161)[0x7f6cec8c4821] - 0x7f6ce3b64710
@ /usr/lib64/libstdc++.so.6:(_Znwm+0x1c)[0x7f6cec83653c] + 0x7f6ce3b94730 0x40001
@ /usr/lib64/libstdc++.so.6:(_ZNSt7__cxx1115basic_stringbufIcSt11char_traitsIcESaIcEE8overflowEi+0x161)[0x7f6cec8c4821] - 0x7f6ce3b74720
@ /usr/lib64/libstdc++.so.6:(_Znwm+0x1c)[0x7f6cec83653c] + 0x7f6ce3bd4740 0x80001
this is gdb stack, when allocation memory size 0x5eec1a
#0 0x00007ffff74665be in malloc () from /lib64/libc.so.6
#1 0x00007ffff79db53c in operator new(unsigned long) () from /usr/lib64/libstdc++.so.6
#2 0x00007ffff7ac9668 in ?? () from /usr/lib64/libstdc++.so.6
#3 0x00007ffff7acc895 in std::__cxx11::basic_stringstream<char, std::char_traits<char>, std::allocator<char> >::str() const & () from /usr/lib64/libstdc++.so.6
#4 0x00000000006f6658 in NetworkServer::SendOriginImage (this=0x8f7460, id="C1527A0090419", image=...) at /home/A306/HebeiSportSystem/RemoteServer/src/NetworkServer.cpp:196
#5 0x00000000007e9805 in TimerThread_sendImageToRemote (stoken=...) at /home/A306/HebeiSportSystem/RemoteServer/run/main.cpp:122
#6 0x00000000007f3167 in std::__invoke_impl<void, void (*)(std::stop_token), std::stop_token> (__f=@0x901dd0: 0x7e976c <TimerThread_sendImageToRemote(std::stop_token)>) at /usr/include/c++/12/bits/invoke.h:61
#7 0x00000000007f30f1 in std::__invoke<void (*)(std::stop_token), std::stop_token> (__fn=@0x901dd0: 0x7e976c <TimerThread_sendImageToRemote(std::stop_token)>) at /usr/include/c++/12/bits/invoke.h:96
#8 0x00000000007f3061 in std::thread::_Invoker<std::tuple<void (*)(std::stop_token), std::stop_token> >::_M_invoke<0ul, 1ul> (this=0x901dc8) at /usr/include/c++/12/bits/std_thread.h:279
#9 0x00000000007f301a in std::thread::_Invoker<std::tuple<void (*)(std::stop_token), std::stop_token> >::operator() (this=0x901dc8) at /usr/include/c++/12/bits/std_thread.h:286
#10 0x00000000007f2ffe in std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)(std::stop_token), std::stop_token> > >::_M_run (this=0x901dc0) at /usr/include/c++/12/bits/std_thread.h:231
#11 0x00007ffff7a06ac3 in ?? () from /usr/lib64/libstdc++.so.6
#12 0x00007ffff7d686ea in start_thread () from /lib64/libpthread.so.0
#13 0x00007ffff74e149f in clone () from /lib64/libc.so.6
this is my code , create temporary std::string
188│ std::stringstream buf;
189│ msgpack::packer<std::stringstream> packer(buf);
190│ packer.pack_str(strlen(image.uuid));
191│ packer.pack_str_body(image.uuid,strlen(image.uuid));
192│ packer.pack_long(image.width);
193│ packer.pack_long(image.height);
194│ packer.pack_bin(image.data_size);
195│ packer.pack_bin_body((const char*)image.data,image.data_size);
196├─> (ptr->second)->send_binary(buf.str()); <----- here
then, std::string allocation memory
#3 0x00007ffff7acc895 in std::__cxx11::basic_stringstream<char, std::char_traits<char>, std::allocator<char> >::str() const & () from /usr/lib64/libstdc++.so.6
but I can't see the memory is deallocation in next time
Thanks. I'll see, if I can reproduce this issue in a minimal example. If so, I can try to fix it. You're using master or one of the releases?
I use master code, the branch hash is 4f3f5deaaa01825c63c83431bfa96ccec195f741
And, Thank you very much
I have made some new discoveries in the process of solving this problem I use a thread to continuously call the 'send_binary()' function, and during the process of sending data out, I added two lines of output to the source code, as shown below
in websocket.h : 632
void do_write()
{
if (sending_buffers_.empty())
{
sending_buffers_.swap(write_buffers_);
spdlog::info("CROW Webscoket -- after swap -- sending_buffers_ : {} , write_buffers_ : {}",sending_buffers_.size(),write_buffers_.size()); // <--- is my add
std::vector<asio::const_buffer> buffers;
buffers.reserve(sending_buffers_.size());
for (auto& s : sending_buffers_)
{
buffers.emplace_back(asio::buffer(s));
}
auto watch = std::weak_ptr<void>{anchor_};
asio::async_write(
adaptor_.socket(), buffers,
[&, watch](const asio::error_code& ec, std::size_t /*bytes_transferred*/) {
if (!ec && !close_connection_)
{
spdlog::info("CROW Webscoket -- async_write -- sending_buffers_ : {} , write_buffers_ : {}",sending_buffers_.size(),write_buffers_.size()); // <-- is my add
sending_buffers_.clear();
if (!write_buffers_.empty())
do_write();
if (has_sent_close_)
close_connection_ = true;
}
else
{
auto anchor = watch.lock();
if (anchor == nullptr) { return; }
sending_buffers_.clear();
close_connection_ = true;
check_destroy();
}
});
}
}
During the program operation, you can see the following output
[2023-08-29 13:55:55.350] [info] CROW Webscoket -- after swap -- sending_buffers_ : 4 , write_buffers_ : 0
[2023-08-29 13:55:55.354] [info] CROW Webscoket -- async_write -- sending_buffers_ : 4 , write_buffers_ : 0
[2023-08-29 13:55:55.372] [info] CROW Webscoket -- after swap -- sending_buffers_ : 2 , write_buffers_ : 0
[2023-08-29 13:55:55.378] [info] CROW Webscoket -- after swap -- sending_buffers_ : 2 , write_buffers_ : 0
[2023-08-29 13:55:55.617] [info] CROW Webscoket -- async_write -- sending_buffers_ : 2 , write_buffers_ : 10
[2023-08-29 13:55:55.618] [info] CROW Webscoket -- after swap -- sending_buffers_ : 10 , write_buffers_ : 0
[2023-08-29 13:55:56.889] [info] CROW Webscoket -- async_write -- sending_buffers_ : 10 , write_buffers_ : 40
[2023-08-29 13:55:56.891] [info] CROW Webscoket -- after swap -- sending_buffers_ : 40 , write_buffers_ : 0
[2023-08-29 13:55:57.348] [info] CROW Webscoket -- async_write -- sending_buffers_ : 2 , write_buffers_ : 66
[2023-08-29 13:55:57.349] [info] CROW Webscoket -- after swap -- sending_buffers_ : 66 , write_buffers_ : 0
[2023-08-29 13:56:01.054] [info] CROW Webscoket -- async_write -- sending_buffers_ : 40 , write_buffers_ : 140
[2023-08-29 13:56:01.058] [info] CROW Webscoket -- after swap -- sending_buffers_ : 140 , write_buffers_ : 0
[2023-08-29 13:56:16.063] [info] CROW Webscoket -- async_write -- sending_buffers_ : 140 , write_buffers_ : 504
[2023-08-29 13:56:16.081] [info] CROW Webscoket -- after swap -- sending_buffers_ : 504 , write_buffers_ : 0
[2023-08-29 13:56:18.982] [info] CROW Webscoket -- async_write -- sending_buffers_ : 66 , write_buffers_ : 728
[2023-08-29 13:56:18.993] [info] CROW Webscoket -- after swap -- sending_buffers_ : 728 , write_buffers_ : 0
The buffer is getting larger, causing the sending speed to become more and more full, which in turn leads to a larger and slower buffer. I think obtaining here can add a blocking call