Support split debug information

[danielocfb]

We should support split debug information.

[danielocfb]

This will likely come once we switch to using gimli.

Edit: Not meaning to imply that it will just magically happen, but more that gimli is a prerequisite.

[danielocfb]

So it appears there is some overlap with https://github.com/libbpf/blazesym/issues/208. Specifically, as per my limited understanding, split DWARF is really a catch-all for:

• objcopy --only-keep-debug [...]; strip --strip-debug [...]; objcopy --add-gnu-debuglink= [...] dance
• dwo
• dwp

We may have to support all three. I believe (but haven't tested), that the first one should already work (as it's not really a different/extended format), though we may need to support following debug links specifically. The other two require special handling.

[r1viollet]

I will add some of the use cases I encounter on split debug in case this helps with prioritization / testing. Starting with some of the simpler strategies, from the same folder. When we install debug symbols in a .NET container, we result in situations where .dbg are next to the library

/usr/share/dotnet/shared/Microsoft.NETCore.App/8.0.0/libcoreclr.so
/usr/share/dotnet/shared/Microsoft.NETCore.App/8.0.0/libcoreclr.so.dbg

The build-ids match, the gnu-debuglink points to libcoreclr.so.dbg

readelf -x .gnu_debuglink /usr/share/dotnet/shared/Microsoft.NETCore.App/8.0.0/libcoreclr.so
Hex dump of section '.gnu_debuglink':
  0x00000000 6c696263 6f726563 6c722e73 6f2e6462 libcoreclr.so.db
  0x00000010 67000000 9c5b2729                   g....[')

[d-e-s-o]

Thanks for the example! Debug link support is indeed most likely the first thing that will come here, as the DWARF format shouldn't be any different from what I understand.

Starting with some of the simpler strategies, from the same folder.

Do you have use cases where the debug information is contained in a different folder? My reading of the "spec" is that only a file name is stored in the link, no path. So in a first version I think the assumption will be that the .debug file has to reside next to the other one. But, if users wanted to have separate debug info located in a different folder, that would need to be made user configurable, I'd think.

[danielocfb]

Oh okay, seems as if readelf is doing the following:

$ readelf -wk /usr/lib64/libc.so.6
readelf: /usr/lib64/libc.so.6: Warning: could not find separate debug file 'libc.so.6-2.37-19.fc38.x86_64.debug'
readelf: /usr/lib64/libc.so.6: Warning: tried: /lib/debug/libc.so.6-2.37-19.fc38.x86_64.debug
readelf: /usr/lib64/libc.so.6: Warning: tried: /usr/lib/debug/usr/libc.so.6-2.37-19.fc38.x86_64.debug
readelf: /usr/lib64/libc.so.6: Warning: tried: /usr/lib/debug//usr/lib64//libc.so.6-2.37-19.fc38.x86_64.debug
readelf: /usr/lib64/libc.so.6: Warning: tried: /usr/lib/debug/libc.so.6-2.37-19.fc38.x86_64.debug
readelf: /usr/lib64/libc.so.6: Warning: tried: /usr/lib64/.debug/libc.so.6-2.37-19.fc38.x86_64.debug
readelf: /usr/lib64/libc.so.6: Warning: tried: /usr/lib64/libc.so.6-2.37-19.fc38.x86_64.debug
readelf: /usr/lib64/libc.so.6: Warning: tried: .debug/libc.so.6-2.37-19.fc38.x86_64.debug
readelf: /usr/lib64/libc.so.6: Warning: tried: libc.so.6-2.37-19.fc38.x86_64.debug
readelf: /usr/lib64/libc.so.6: Warning: tried: DEBUGINFOD_URLS=
Contents of the .gnu_debuglink section:

  Separate debug info file: libc.so.6-2.37-19.fc38.x86_64.debug
  CRC value: 0xa55cda70

So it's looking in various "well known" directories...great.

[danielocfb]

FYI, Debug link support is coming with https://github.com/libbpf/blazesym/pull/665

[danielocfb]

Bumping this to P1 as we will need debug fission support soon.