Fast-DDS icon indicating copy to clipboard operation
Fast-DDS copied to clipboard

explosive cpu usage using share memory transport [13717]

Open happykkkkai opened this issue 3 years ago • 4 comments

hi. I try to test dds inter-process communication using share memory transport, which is bidirectional, that is say each process has one publisher to publish data to another process and one subscriber to receive data. Usually the cpu loading is about 20% for one process. However, when i kill one process manually, the cpu loading of another process maybe explosive increase to about 100%.

The system i used is [Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz for 64 cores] and the os is [Linux version 5.11.0-41-generic (buildd@lgw01-amd64-005) (gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0, GNU ld (GNU Binutils for Ubuntu) 2.34) #45~20.04.1-Ubuntu SMP].

I try to use perf to locator the hot spot of cpu loading, and the most one is the thread for DDS share memory listening, as showed follow:

Thread 28 (Thread 0x7fe5fc1b1700 (LWP 764522)):
#0  0x00007fe613de10b1 in __GI___pthread_mutex_lock (mutex=0x7fe614a5a990 <_rtld_global+2352>) at ../nptl/pthread_mutex_lock.c:115
#1  0x00007fe613bc8231 in __GI___dl_iterate_phdr (callback=0x7fe613c6a5f0, data=0x7fe5fc1b01c0) at dl-iteratephdr.c:40
#2  0x00007fe613c6b6c1 in _Unwind_Find_FDE () from /lib/x86_64-linux-gnu/libgcc_s.so.1
#3  0x00007fe613c67868 in ?? () from /lib/x86_64-linux-gnu/libgcc_s.so.1
#4  0x00007fe613c68b9d in ?? () from /lib/x86_64-linux-gnu/libgcc_s.so.1
#5  0x00007fe613c695aa in _Unwind_Resume () from /lib/x86_64-linux-gnu/libgcc_s.so.1
#6  0x00005651706cb8c0 in eprosima::fastdds::rtps::SharedMemManager::Listener::pop() ()
#7  0x00005651706cbbc5 in eprosima::fastdds::rtps::SharedMemChannelResource::Receive(eprosima::fastrtps::rtps::Locator_t&) ()
#8  0x00005651706b610c in eprosima::fastdds::rtps::SharedMemChannelResource::perform_listen_operation(eprosima::fastrtps::rtps::Locator_t) ()
#9  0x00005651706a6abc in std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (eprosima::fastdds::rtps::SharedMemChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::SharedMemChannelResource*, eprosima::fastrtps::rtps::Locator_t> > >::_M_run() ()
#10 0x00007fe613ecede4 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#11 0x00007fe613dde609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#12 0x00007fe613b88293 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

happykkkkai avatar Jan 11 '22 07:01 happykkkkai

the version i used is Fast-DDS-2.3.3

happykkkkai avatar Jan 11 '22 07:01 happykkkkai

#2237 might also be related

JLBuenoLopez avatar Feb 01 '22 10:02 JLBuenoLopez

how to disable shm transport?

qpc001 avatar Feb 18 '22 11:02 qpc001

You can find the answer in Fast DDS documentation. You must disable the builtin_transports and enable an alternative transport (the link leads to the UDPv4 transport, but you may enable any transport you are interested in).

JLBuenoLopez avatar Feb 18 '22 11:02 JLBuenoLopez

According to our CONTRIBUTING.md guidelines, I am closing this issue due to inactivity. Please, feel free to reopen it if necessary.

Mario-DL avatar Mar 24 '23 08:03 Mario-DL