TarsCpp icon indicating copy to clipboard operation
TarsCpp copied to clipboard

tarscpp 2.4.6版本上跑不停反复启动脚本发现coredump,麻烦确认下最新版本是否已经解决

Open zjj-118 opened this issue 3 years ago • 4 comments

跑反复启动脚本发现tarsAdminRegistry模块频繁出现coredump,基本每次都是在communicator释放时出现的。具体当时的Log信息如下:

[root@gateway-tars-0 bin]# gdb tarsAdminRegistry core.tarsAdminRegist.0.3d8e8875faea42d882c5d783238b10dc.3978.1606457834000000 GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-119.el7 Copyright (C) 2013 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". For bug reporting instructions, please see: http://www.gnu.org/software/gdb/bugs/... Reading symbols from /usr/local/app/tars/tarsAdminRegistry/bin/tarsAdminRegistry...done.

warning: core file may not match specified executable file. [New LWP 1562] [New LWP 1563] [New LWP 1557] [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Core was generated by /usr/local/app/tars/tarsAdminRegistry/bin/tarsAdminRegistry --config=/usr/local'. Program terminated with signal 11, Segmentation fault. #0 0x00007f74433f83f7 in __memcpy_sse2 () from /lib64/libc.so.6 Missing separate debuginfos, use: debuginfo-install glibc-2.17-260.el7.x86_64 libgcc-4.8.5-36.el7.x86_64 libstdc++-4.8.5-36.el7.x86_64 (gdb) bt #0 0x00007f74433f83f7 in __memcpy_sse2 () from /lib64/libc.so.6 #1 0x00007f7443d077b4 in std::string::append(std::string const&) () from /lib64/libstdc++.so.6 #2 0x0000000000681cd5 in std::operator+<char, std::char_traits<char>, std::allocator<char> >(std::basic_string<char, std::char_traits<char>, std::allocator<char> >&&, std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) (__lhs=<unknown type in /usr/local/app/tars/tarsAdminRegistry/bin/tarsAdminRegistry, CU 0x3d52b, DIE 0xc06f3>, __rhs="2.4.5", '\000' <repeats 11 times>, "!\004\000\000\000\000\000\000\220\251=\001\000\000\000\000\340\313>\001", '\000' <repeats 20 times>, "@R\360\000\000\000\000\000(\250\214\000\000\000\000\000\004\000\000\000\000\000\000\000\354\250\306IRg\374\231R\360\000\000\000\000\000(\250\214\000\000\000\000\000\004\000\000\000\000\000\000\000\354\250\246ldg\374\231hR\360\000\000\000\000\000(\250\214\000\000\000\000\000\004\000\000\000\000\000\000\000\354\250\246ldg\374\231pR\360\000\000\000\000\000(\250\214\000\000\000\000\000\004\000\000\000\000\000\000\000\354\250\306IRg\374\231\210R\360\000\000\000\000\000(\250\214\000\000\000\000\000"...<Address 0x1422000 out of bounds>) at /opt/rh/devtoolset-7/root/usr/include/c++/7/bits/basic_string.h:5943 #3 0x0000000000780d83 in tars::AdapterProxy::initStatHead (this=0x7f743c004130) at /root/sync/NCS_V100R003/tars/TarsFramework/tarscpp/servant/libservant/AdapterProxy.cpp:120 #4 0x00000000007809a7 in tars::AdapterProxy::AdapterProxy (this=0x7f743c004130, pObjectProxy=0x140c6a0, ep=..., pCom=0x13de800) at /root/sync/NCS_V100R003/tars/TarsFramework/tarscpp/servant/libservant/AdapterProxy.cpp:82 Python Exception <type 'exceptions.ValueError'> Cannot find type const std::set<tars::EndpointInfo, std::lesstars::EndpointInfo, std::allocatortars::EndpointInfo >::_Rep_type: Python Exception <type 'exceptions.ValueError'> Cannot find type const std::set<tars::EndpointInfo, std::lesstars::EndpointInfo, std::allocatortars::EndpointInfo >::_Rep_type: #5 0x000000000079fe1e in tars::EndpointManager::updateEndpoints (this=0x140c7f0, active=std::set with 1 elements, inactive=std::set with 0 elements) at /root/sync/NCS_V100R003/tars/TarsFramework/tarscpp/servant/libservant/EndpointManager.cpp:746 Python Exception <type 'exceptions.ValueError'> Cannot find type const std::set<tars::EndpointInfo, std::lesstars::EndpointInfo, std::allocatortars::EndpointInfo >::_Rep_type: Python Exception <type 'exceptions.ValueError'> Cannot find type const std::set<tars::EndpointInfo, std::lesstars::EndpointInfo, std::allocatortars::EndpointInfo >::_Rep_type: #6 0x00000000007a0478 in tars::EndpointManager::notifyEndpoints (this=0x140c7f0, active=std::set with 1 elements, inactive=std::set with 0 elements, bNotify=false) at /root/sync/NCS_V100R003/tars/TarsFramework/tarscpp/servant/libservant/EndpointManager.cpp:811 #7 0x000000000079e658 in tars::QueryEpBase::doEndpoints (this=0x140c7f0, activeEp=std::vector of length 1, capacity 1 = {...}, inactiveEp=std::vector of length 0, capacity 0, iRet=0, bSync=false) at /root/sync/NCS_V100R003/tars/TarsFramework/tarscpp/servant/libservant/EndpointManager.cpp:603 #8 0x000000000079b926 in tars::QueryEpBase::callback_findObjectByIdInSameGroup (this=0x140c7f0, ret=0, activeEp=std::vector of length 1, capacity 1 = {...}, inactiveEp=std::vector of length 0, capacity 0) at /root/sync/NCS_V100R003/tars/TarsFramework/tarscpp/servant/libservant/EndpointManager.cpp:90 #9 0x00000000007a7796 in tars::QueryFPrxCallback::onDispatch (this=0x140c7f0, msg=...) at /root/sync/NCS_V100R003/tars/TarsFramework/tarscpp/servant/servant/QueryF.h:196 #10 0x000000000075bce6 in tars::ServantProxyCallback::dispatch (this=0x140c7f0, msg=...) at /root/sync/NCS_V100R003/tars/TarsFramework/tarscpp/servant/libservant/ServantProxy.cpp:167 #11 0x0000000000785062 in tars::AdapterProxy::finishInvoke (this=0x140a150, msg=0x7f743c000f60) at /root/sync/NCS_V100R003/tars/TarsFramework/tarscpp/servant/libservant/AdapterProxy.cpp:871 #12 0x0000000000784a44 in tars::AdapterProxy::finishInvoke_parallel (this=0x140a150, rsp=std::shared_ptr (count 2, weak 0) 0x7f743c003440) at /root/sync/NCS_V100R003/tars/TarsFramework/tarscpp/servant/libservant/AdapterProxy.cpp:743 #13 0x0000000000784c4a in tars::AdapterProxy::finishInvoke (this=0x140a150, rsp=std::shared_ptr (count 2, weak 0) 0x7f743c003440) at /root/sync/NCS_V100R003/tars/TarsFramework/tarscpp/servant/libservant/AdapterProxy.cpp:756 #14 0x000000000077c84d in tars::Transceiver::finishInvoke (this=0x140a600, rsp=std::shared_ptr (count 2, weak 0) 0x7f743c003440) at /root/sync/NCS_V100R003/tars/TarsFramework/tarscpp/servant/libservant/Transceiver.cpp:351 #15 0x000000000077d641 in tars::TcpTransceiver::doResponse (this=0x140a600) at /root/sync/NCS_V100R003/tars/TarsFramework/tarscpp/servant/libservant/Transceiver.cpp:648 #16 0x000000000071b38f in tars::CommunicatorEpoll::handleInputImp (this=0x1401a50, pTransceiver=0x140a600) at /root/sync/NCS_V100R003/tars/TarsFramework/tarscpp/servant/libservant/CommunicatorEpoll.cpp:163 #17 0x000000000071b89c in tars::CommunicatorEpoll::handle (this=0x1401a50, pFDInfo=0x140a688, ev=...) at /root/sync/NCS_V100R003/tars/TarsFramework/tarscpp/servant/libservant/CommunicatorEpoll.cpp:277 #18 0x000000000071d5a6 in tars::CommunicatorEpoll::run (this=0x1401a50) at /root/sync/NCS_V100R003/tars/TarsFramework/tarscpp/servant/libservant/CommunicatorEpoll.cpp:468 #19 0x0000000000814c0f in tars::TC_Thread::threadEntry (pThread=0x1401a50) at /root/sync/NCS_V100R003/tars/TarsFramework/tarscpp/util/src/tc_thread.cpp:92 #20 0x00000000008154f3 in std::__invoke_impl<void, void ()(tars::TC_Thread), tars::TC_Thread*>(std::__invoke_other, void (&&)(tars::TC_Thread), tars::TC_Thread*&&) ( __f=<unknown type in /usr/local/app/tars/tarsAdminRegistry/bin/tarsAdminRegistry, CU 0xb44d3f, DIE 0xb4ffbd>, __args#0=<unknown type in /usr/local/app/tars/tarsAdminRegistry/bin/tarsAdminRegistry, CU 0xb44d3f, DIE 0xb4ffd8>) at /opt/rh/devtoolset-7/root/usr/include/c++/7/bits/invoke.h:60 #21 0x000000000081521c in std::__invoke<void ()(tars::TC_Thread), tars::TC_Thread*>(void (&&)(tars::TC_Thread), tars::TC_Thread*&&) ( __fn=<unknown type in /usr/local/app/tars/tarsAdminRegistry/bin/tarsAdminRegistry, CU 0xb44d3f, DIE 0xb5061d>, __args#0=<unknown type in /usr/local/app/tars/tarsAdminRegistry/bin/tarsAdminRegistry, CU 0xb44d3f, DIE 0xb50638>) at /opt/rh/devtoolset-7/root/usr/include/c++/7/bits/invoke.h:95 #22 0x000000000081578d in std::thread::_Invoker<std::tuple<void ()(tars::TC_Thread), tars::TC_Thread*> >::_M_invoke<0ul, 1ul> (this=0x13dfc18) at /opt/rh/devtoolset-7/root/usr/include/c++/7/thread:234 #23 0x0000000000815743 in std::thread::_Invoker<std::tuple<void ()(tars::TC_Thread), tars::TC_Thread*> >::operator() (this=0x13dfc18) at /opt/rh/devtoolset-7/root/usr/include/c++/7/thread:243 #24 0x0000000000815722 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<void ()(tars::TC_Thread), tars::TC_Thread*> > >::_M_run (this=0x13dfc10) at /opt/rh/devtoolset-7/root/usr/include/c++/7/thread:186 #25 0x00000000008c4fef in execute_native_thread_routine () #26 0x00007f744415bdd5 in start_thread () from /lib64/libpthread.so.0 #27 0x00007f7443461ead in clone () from /lib64/libc.so.6 (gdb) frame 2 #2 0x0000000000681cd5 in std::operator+<char, std::char_traits, std::allocator >(std::basic_string<char, std::char_traits, std::allocator >&&, std::basic_string<char, std::char_traits, std::allocator > const&) (__lhs=<unknown type in /usr/local/app/tars/tarsAdminRegistry/bin/tarsAdminRegistry, CU 0x3d52b, DIE 0xc06f3>, __rhs="2.4.5", '\000' <repeats 11 times>, "!\004\000\000\000\000\000\000\220\251=\001\000\000\000\000\340\313>\001", '\000' <repeats 20 times>, "@R\360\000\000\000\000\000(\250\214\000\000\000\000\000\004\000\000\000\000\000\000\000\354\250\306IRg\374\231R\360\000\000\000\000\000(\250\214\000\000\000\000\000\004\000\000\000\000\000\000\000\354\250\246ldg\374\231hR\360\000\000\000\000\000(\250\214\000\000\000\000\000\004\000\000\000\000\000\000\000\354\250\246ldg\374\231pR\360\000\000\000\000\000(\250\214\000\000\000\000\000\004\000\000\000\000\000\000\000\354\250\306IRg\374\231\210R\360\000\000\000\000\000(\250\214\000\000\000\000\000"...<Address 0x1422000 out of bounds>) at /opt/rh/devtoolset-7/root/usr/include/c++/7/bits/basic_string.h:5943 5943 /opt/rh/devtoolset-7/root/usr/include/c++/7/bits/basic_string.h: No such file or directory. (gdb) p ClientConfig::TarsVersion $1 = "2.4.5", '\000' <repeats 11 times>, "!\004\000\000\000\000\000\000\220\251=\001\000\000\000\000\340\313>\001", '\000' <repeats 20 times>, "@R\360\000\000\000\000\000(\250\214\000\000\000\000\000\004\000\000\000\000\000\000\000\354\250\306IRg\374\231R\360\000\000\000\000\000(\250\214\000\000\000\000\000\004\000\000\000\000\000\000\000\354\250\246ldg\374\231hR\360\000\000\000\000\000(\250\214\000\000\000\000\000\004\000\000\000\000\000\000\000\354\250\246ldg\374\231pR\360\000\000\000\000\000(\250\214\000\000\000\000\000\004\000\000\000\000\000\000\000\354\250\306IRg\374\231\210R\360\000\000\000\000\000(\250\214\000\000\000\000\000"...<Address 0x1422000 out of bounds> (gdb) p *(std::string *)__lhs $2 = "tars.tarsAdminRegistry@" (gdb) p __rhs $3 = "2.4.5", '\000' <repeats 11 times>, "!\004\000\000\000\000\000\000\220\251=\001\000\000\000\000\340\313>\001", '\000' <repeats 20 times>, "@R\360\000\000\000\000\000(\250\214\000\000\000\000\000\004\000\000\000\000\000\000\000\354\250\306IRg\374\231`R\360\000\000\000\000\000(\250\214\000\000\000\000\000\004\000\000\000\000\000\000\000\354\250\246ldg\374\231hR\360\000\000\000\000\000(\250\214\000\000\000\000\000\004\000\000\000\000\000\000\000\354\250\246ldg\374\231pR\360\000\000\000\000\000(\250\214\000\000\000\000\000\004\000\000\000\000\000\000\000\354\250\306IRg\374\231\210R\360\000\000\000\000\000(\250\214\000\000\000\000\000"...<Address 0x1422000 out of bounds> (gdb) info threads Id Target Id Frame 3 Thread 0x7f7444587780 (LWP 1557) 0x00007f744415cf47 in pthread_join () from /lib64/libpthread.so.0 2 Thread 0x7f7440b5e700 (LWP 1563) 0x00007f7444162e3d in nanosleep () from /lib64/libpthread.so.0

  • 1 Thread 0x7f744135f700 (LWP 1562) 0x00007f74433f83f7 in __memcpy_sse2 () from /lib64/libc.so.6 (gdb) thread 3 [Switching to thread 3 (Thread 0x7f7444587780 (LWP 1557))] #0 0x00007f744415cf47 in pthread_join () from /lib64/libpthread.so.0 (gdb) bt #0 0x00007f744415cf47 in pthread_join () from /lib64/libpthread.so.0 #1 0x00007f7443cfde37 in std::thread::join() () from /lib64/libstdc++.so.6 #2 0x00000000008149b3 in tars::TC_ThreadControl::join (this=0x7ffc4d5c1968) at /root/sync/NCS_V100R003/tars/TarsFramework/tarscpp/util/src/tc_thread.cpp:38 #3 0x0000000000715536 in tars::Communicator::terminate (this=0x13de800) at /root/sync/NCS_V100R003/tars/TarsFramework/tarscpp/servant/libservant/Communicator.cpp:536 #4 0x000000000071089b in tars::Communicator::~Communicator (this=0x13de800, __in_chrg=) at /root/sync/NCS_V100R003/tars/TarsFramework/tarscpp/servant/libservant/Communicator.cpp:73 #5 0x0000000000710944 in tars::Communicator::~Communicator (this=0x13de800, __in_chrg=) at /root/sync/NCS_V100R003/tars/TarsFramework/tarscpp/servant/libservant/Communicator.cpp:78 #6 0x000000000063a075 in tars::TC_HandleBase::decRef (this=0x13de800) at /root/sync/NCS_V100R003/tars/TarsFramework/tarscpp/util/include/util/tc_autoptr.h:79 #7 0x0000000000691a9d in tars::TC_AutoPtrtars::Communicator::~TC_AutoPtr (this=0xf04fb8 tars::Application::_communicator, __in_chrg=) at /root/sync/NCS_V100R003/tars/TarsFramework/tarscpp/util/include/util/tc_autoptr.h:227 #8 0x00007f744339db69 in __run_exit_handlers () from /lib64/libc.so.6 #9 0x00007f744339dbb7 in exit () from /lib64/libc.so.6 #10 0x00000000006ec752 in tars::Application::main (this=0xf049e0 <g_app>, option=...) at /root/sync/NCS_V100R003/tars/TarsFramework/tarscpp/servant/libservant/Application.cpp:849 #11 0x00000000006ea918 in tars::Application::main (this=0xf049e0 <g_app>, argc=2, argv=0x7ffc4d5c2368) at /root/sync/NCS_V100R003/tars/TarsFramework/tarscpp/servant/libservant/Application.cpp:690 #12 0x00000000006b0d08 in main (argc=2, argv=0x7ffc4d5c2368) at /root/sync/NCS_V100R003/tars/TarsFramework/AdminRegistryServer/AdminRegistryServer.cpp:133 (gdb) thread 2 [Switching to thread 2 (Thread 0x7f7440b5e700 (LWP 1563))] #0 0x00007f7444162e3d in nanosleep () from /lib64/libpthread.so.0 (gdb) bt #0 0x00007f7444162e3d in nanosleep () from /lib64/libpthread.so.0 #1 0x00000000006fac01 in std::this_thread::sleep_for<long, std::ratio<1l, 1000l> > (__rtime=...) at /opt/rh/devtoolset-7/root/usr/include/c++/7/thread:373 #2 0x000000000081955a in tars::TC_TimeProvider::run (this=0x7f743c000ba0) at /root/sync/NCS_V100R003/tars/TarsFramework/tarscpp/util/src/tc_timeprovider.cpp:140 #3 0x0000000000819ca4 in std::__invoke_impl<void, void (tars::TC_TimeProvider::)(), tars::TC_TimeProvider>(std::__invoke_memfun_deref, void (tars::TC_TimeProvider::&&)(), tars::TC_TimeProvider&&) ( __f=<unknown type in /usr/local/app/tars/tarsAdminRegistry/bin/tarsAdminRegistry, CU 0xb6f741, DIE 0xb766d8>, __t=<unknown type in /usr/local/app/tars/tarsAdminRegistry/bin/tarsAdminRegistry, CU 0xb6f741, DIE 0xb766e6>) at /opt/rh/devtoolset-7/root/usr/include/c++/7/bits/invoke.h:73 #4 0x000000000081999a in std::__invoke<void (tars::TC_TimeProvider::)(), tars::TC_TimeProvider>(void (tars::TC_TimeProvider::&&)(), tars::TC_TimeProvider&&) ( __fn=<unknown type in /usr/local/app/tars/tarsAdminRegistry/bin/tarsAdminRegistry, CU 0xb6f741, DIE 0xb7700e>, __args#0=<unknown type in /usr/local/app/tars/tarsAdminRegistry/bin/tarsAdminRegistry, CU 0xb6f741, DIE 0xb77029>) at /opt/rh/devtoolset-7/root/usr/include/c++/7/bits/invoke.h:95 #5 0x0000000000819f6b in std::thread::_Invoker<std::tuple<void (tars::TC_TimeProvider::)(), tars::TC_TimeProvider> >::_M_invoke<0ul, 1ul> (this=0x7f743c000c08) at /opt/rh/devtoolset-7/root/usr/include/c++/7/thread:234 #6 0x0000000000819f21 in std::thread::_Invoker<std::tuple<void (tars::TC_TimeProvider::)(), tars::TC_TimeProvider> >::operator() (this=0x7f743c000c08) at /opt/rh/devtoolset-7/root/usr/include/c++/7/thread:243 #7 0x0000000000819f00 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (tars::TC_TimeProvider::)(), tars::TC_TimeProvider> > >::_M_run (this=0x7f743c000c00) at /opt/rh/devtoolset-7/root/usr/include/c++/7/thread:186 #8 0x00000000008c4fef in execute_native_thread_routine () #9 0x00007f744415bdd5 in start_thread () from /lib64/libpthread.so.0 #10 0x00007f7443461ead in clone () from /lib64/libc.so.6

zjj-118 avatar Nov 30 '20 00:11 zjj-118

最新的版本修复了退出析构的问题, 可以更新一下, 看看是否还会出现

ruanshudong avatar Dec 08 '20 01:12 ruanshudong

好的。

zjj-118 avatar Dec 10 '20 05:12 zjj-118

*** 这个问题在最新的2.4.14上还是存在,core.dump的调用栈还是一样的。***

zjj-118 avatar Dec 14 '20 08:12 zjj-118

只是现在出现频率不高了。

zjj-118 avatar Dec 14 '20 09:12 zjj-118