Problem when linked with jemalloc or tcmalloc.

[czlhs]

coz run program linked with tcmalloc will deadlock, run program linked with jeamlloc will be Segment fault.

the stack with tcmalloc.

#0  0x00007ffff6957e30 in __nanosleep_nocancel () from /lib64/libpthread.so.0
#1  0x00000000005d25b9 in base::internal::SpinLockDelay (w=0xb295c8 <tcmalloc::Static::pageheap_lock_>, value=2, loop=56075)
    at ./src/base/spinlock_linux-inl.h:86
#2  0x00000000005d2544 in SpinLock::SlowLock (this=this@entry=0xb295c8 <tcmalloc::Static::pageheap_lock_>) at src/base/spinlock.cc:119
#3  0x00000000005cdf88 in SpinLock::Lock (this=<optimized out>) at src/base/spinlock.h:71
#4  SpinLockHolder::SpinLockHolder (l=<optimized out>, this=<synthetic pointer>) at src/base/spinlock.h:133
#5  tcmalloc::ThreadCache::InitModule () at src/thread_cache.cc:286
#6  0x00000000005ce07d in tcmalloc::ThreadCache::CreateCacheIfNecessary () at src/thread_cache.cc:329
#7  0x000000000068546d in tcmalloc::ThreadCache::GetCache () at src/thread_cache.h:442
#8  (anonymous namespace)::do_malloc (size=32) at src/tcmalloc.cc:1350
#9  (anonymous namespace)::do_malloc_or_cpp_alloc (size=32) at src/tcmalloc.cc:1375
#10 (anonymous namespace)::do_calloc (elem_size=<optimized out>, n=<optimized out>) at src/tcmalloc.cc:1388
#11 tc_calloc (n=<optimized out>, elem_size=<optimized out>) at src/tcmalloc.cc:1942
#12 0x00007ffff7286620 in _dlerror_run () from /lib64/libdl.so.2
#13 0x00007ffff7286128 in dlsym () from /lib64/libdl.so.2
#14 0x00007ffff7b82cc7 in resolve_sigprocmask (how=2, set=0x7fffffffd4d0, oldset=0x7fffffffd570) at real.cpp:97
#15 0x00007ffff7b75b1a in sigprocmask (how=2, set=<optimized out>, oldset=0x7fffffffd570) at libcoz.cpp:411
#16 0x00000000005e7f33 in _ULx86_64_init ()
#17 0x00000000005e19f9 in _ULx86_64_init_local ()
#18 0x00000000005d188a in GetStackTrace_libunwind (result=0xb63010, max_depth=30, skip_count=5) at src/stacktrace_libunwind-inl.h:116
#19 0x00000000005d23d4 in GetStackTrace (result=result@entry=0xb63010, max_depth=max_depth@entry=30, skip_count=skip_count@entry=3)
    at src/stacktrace.cc:295
#20 0x00000000005cbee6 in tcmalloc::RecordGrowth (growth=1048576) at src/page_heap.cc:618
#21 tcmalloc::PageHeap::GrowHeap (this=0x988dc0 <tcmalloc::Static::pageheap_>, n=<optimized out>) at src/page_heap.cc:644
#22 0x00000000005cc1e3 in tcmalloc::PageHeap::New (this=0x988dc0 <tcmalloc::Static::pageheap_>, n=n@entry=1) at src/page_heap.cc:154
#23 0x00000000005ca185 in tcmalloc::CentralFreeList::Populate (this=this@entry=0xb13ac0 <tcmalloc::Static::central_cache_+31616>)
    at src/central_freelist.cc:329
#24 0x00000000005ca388 in tcmalloc::CentralFreeList::FetchFromOneSpansSafe (this=0xb13ac0 <tcmalloc::Static::central_cache_+31616>, N=1,
    start=0x7fffffffdfa0, end=0x7fffffffdfa8) at src/central_freelist.cc:284
#25 0x00000000005ca414 in tcmalloc::CentralFreeList::RemoveRange (this=0xb13ac0 <tcmalloc::Static::central_cache_+31616>,
    start=start@entry=0x7fffffffdfa0, end=end@entry=0x7fffffffdfa8, N=1) at src/central_freelist.cc:264
#26 0x00000000005cd350 in tcmalloc::ThreadCache::FetchFromCentralCache (this=this@entry=0xba3000, cl=cl@entry=26, byte_size=byte_size@entry=576,
    oom_handler=oom_handler@entry=0x5c4ee0 <(anonymous namespace)::nop_oom_handler(size_t)>) at src/thread_cache.cc:126
#27 0x000000000068678b in tcmalloc::ThreadCache::Allocate (oom_handler=0x5c4ee0 <(anonymous namespace)::nop_oom_handler(size_t)>, cl=26, size=576,
    this=<optimized out>) at src/thread_cache.h:380
#28 (anonymous namespace)::do_malloc (size=568) at src/tcmalloc.cc:1367
#29 tcmalloc::do_allocate_full<tcmalloc::malloc_oom> (size=568) at src/tcmalloc.cc:1758
#30 tcmalloc::allocate_full_malloc_oom (size=568) at src/tcmalloc.cc:1774
#31 0x00007ffff65f158d in __fopen_internal () from /lib64/libc.so.6
#32 0x00007ffff4a76a16 in selinuxfs_exists () from /lib64/libselinux.so.1
#33 0x00007ffff4a6ece8 in init_lib () from /lib64/libselinux.so.1
#34 0x00007ffff7dea503 in _dl_init_internal () from /lib64/ld-linux-x86-64.so.2
#35 0x00007ffff7ddc1aa in _dl_start_user () from /lib64/ld-linux-x86-64.so.2

libc => Allocate( tcmalloc(1)) => sigprocmask => libcoz => libdl(dlsym) => tc_calloc(tcmalloc(2)), So tcmalloc(1) has got the lock, and tcmalloc(2) here will deadlock.

the stacktrace of jemalloc:

#236 imalloc_init_check (dopts=<synthetic pointer>, sopts=<synthetic pointer>) at src/jemalloc.c:2229
#237 imalloc (dopts=<synthetic pointer>, sopts=<synthetic pointer>) at src/jemalloc.c:2260
#238 calloc (num=1, size=32) at src/jemalloc.c:2494
#239 0x00007ffff7286620 in _dlerror_run () from /lib64/libdl.so.2
#240 0x00007ffff7286128 in dlsym () from /lib64/libdl.so.2
#241 0x00007ffff7b83430 in resolve_pthread_mutex_lock (mutex=0x9ed8c0 <init_lock+64>) at real.cpp:169
#242 0x00007ffff7b77ade in pthread_mutex_lock (mutex=mutex@entry=0x9ed8c0 <init_lock+64>) at libcoz.cpp:268
#243 0x000000000061db7b in malloc_mutex_lock_final (mutex=0x9ed880 <init_lock>) at include/jemalloc/internal/mutex.h:155
#244 je_malloc_mutex_lock_slow (mutex=mutex@entry=0x9ed880 <init_lock>) at src/mutex.c:85
#245 0x00000000005c89ea in malloc_mutex_lock (mutex=0x9ed880 <init_lock>, tsdn=0x0) at include/jemalloc/internal/mutex.h:221
#246 malloc_init_hard () at src/jemalloc.c:1739
#247 0x00000000005d3a97 in malloc_init () at src/jemalloc.c:223

jemalloc_init => pthread_mutex_lock => libcoz.so => dlsym => jemalloc_alloc => jemalloc_init ... it will infinite loop here until stack overflow.