libbacktrace icon indicating copy to clipboard operation
libbacktrace copied to clipboard

differences with addr2line

Open alk opened this issue 1 year ago • 3 comments

As part of trying to test my fix for binary search thingy I made a small tool that lets me compare addr2line output and what libbacktrace provides. It is at https://github.com/alk/gperftools/tree/wip-compare-with-a2l in case you care enough to look.

It discovers several interesting differences. Some might be worth addressing.

For example here is one such difference:

pc: 0x00021068 vs 0x00021068
NM names: _ZN8tcmalloc19CentralCacheLockAllEv
location sizes mismatch 5 vs 1
a2l 0: _ZNSt13__atomic_baseIiE21compare_exchange_weakERiiSt12memory_orderS2_ /usr/include/c++/14/bits/atomic_base.h:536
mpp 0: _ZN8tcmalloc19CentralCacheLockAllEv /usr/include/c++/14/bits/atomic_base.h:536
a2l 1: _ZNSt13__atomic_baseIiE21compare_exchange_weakERiiSt12memory_order /usr/include/c++/14/bits/atomic_base.h:555
a2l 2: _ZN8SpinLock4LockEv /home/me/src/External/gperftools/src/base/spinlock.h:59
a2l 3: _ZN8tcmalloc15CentralFreeList4LockEv /home/me/src/External/gperftools/src/central_freelist.h:81
a2l 4: _ZN8tcmalloc19CentralCacheLockAllEv /home/me/src/External/gperftools/src/static_vars.cc:99

'mpp' line is what libbacktrace gave and a2l is from addr2line (with -f -i -a given). I am not demangling on both ends.

Disasm of the relevant code tells me that addr2line is right. Definitely looks like we're in the middle of taking spinlock in atomic op.

   21063:       b9 01 00 00 00          mov    $0x1,%ecx
   21068:       f0 0f b1 0b             lock cmpxchg %ecx,(%rbx)
   2106c:       75 1f                   jne    2108d <_ZN8tcmalloc19CentralCacheLockAllEv+0x6d>

Also notably that libbacktrace is giving me right "outer" function. But "fails" at inlining (which imho would be okay-ish) but then gives me wrong filename:line no information.

I have couple questions:

  • is it worth addressing those kinds of differences?
  • if so, do you mind giving me any pointers how/where to look? I have time I can offer to help with these.

alk avatar Oct 07 '24 18:10 alk

Sure, it's worth investigating these. Thanks. Perhaps the place to start is to look at the DWARF information, such as with readelf --debug or dwarfdump, and figure out where the debug info is coming from. Then look at libbacktrace/dwarf.c to see what is going wrong. Thanks.

ianlancetaylor avatar Oct 07 '24 18:10 ianlancetaylor

So this specific case is another instance of "binary search trouble". Just for function ranges. Well, or subset. In this instance I got range with length 0 which we could simply skip, that caused the trouble. What are your thoughts on my approach for the fix I posted?

alk avatar Oct 08 '24 18:10 alk

Sorry for slow replies, I've gotten swamped with my actual job.

ianlancetaylor avatar Oct 09 '24 17:10 ianlancetaylor