[Fix] Symbol mapping issue when we have multiple executable segment

[yinghai]

It turns out that we can have multiple executable segment for python binaries

➜  py-spy git:(master) ✗ grep libpython3 /proc/3033958/maps
7b9d5718a000-7b9d5718b000 r-xp 0065e000 09:00 271849319                  /tmp/yinghai/cpython-3.12.9-linux-x86_64-gnu/lib/libpython3.12.so.1.0
7b9d580f1000-7b9d58311000 r--p 00000000 09:00 271849319                  /tmp/yinghai/cpython-3.12.9-linux-x86_64-gnu/lib/libpython3.12.so.1.0
7b9d58311000-7b9d5902e000 r-xp 00220000 09:00 271849319                  /tmp/yinghai/cpython-3.12.9-linux-x86_64-gnu/lib/libpython3.12.so.1.0
7b9d5902e000-7b9d5962a000 r--p 00f3d000 09:00 271849319                  /tmp/yinghai/cpython-3.12.9-linux-x86_64-gnu/lib/libpython3.12.so.1.0
7b9d5962a000-7b9d5974f000 r--p 01538000 09:00 271849319                  /tmp/yinghai/cpython-3.12.9-linux-x86_64-gnu/lib/libpython3.12.so.1.0
7b9d5974f000-7b9d598e6000 rw-p 0165d000 09:00 271849319                  /tmp/yinghai/cpython-3.12.9-linux-x86_64-gnu/lib/libpython3.12.so.1.0

Note that both segement 7b9d5718a000 and 7b9d58311000 are executable but first segment has a lower address but a higher offset than the second one. In the code to parse binary, we find the first executable with lowerest address but in symbol map resolution, we use elf in linux to find the first executable with lowest offset to get absolute address of a symbol. https://github.com/benfred/py-spy/blob/1fa3a6ded252d7c1c0ff974a4fcd1af67a1577cf/src/binary_parser.rs#L139-L145

This inconsistency causes the symbol address to wrong. Hence py-spy will fail to find python context of the process. Typical error message is

Failed to find python version from target process

We have a few of such issues (https://github.com/benfred/py-spy/issues/756, https://github.com/benfred/py-spy/issues/550), which could be related although there are other reasons that can lead to this.

The fix here is to scan all the executable segments and pick the lowest offset one to parse binary so that we are consistent.

[korniltsev]

Can you please share the libpython binary or a way to reproduce / obtain it?

[korniltsev]

Nit: It would be nice to have a regression test.

[yinghai]

I actually don't think this is library specific. It tends to happen on the spawned subprocess. It's just the binaries go remapped and the segments are reordered in terms of memory address vs offset.

[petersalas]

FWIW I ran into this issue with latest py-spy, saw similar segments as are described above, and confirmed that @yinghai's branch worked perfectly.

[fleimgruber]

@yinghai I wanted to check out your branch on Windows OS and get a compile error:

error[E0609]: no field `offset` on type `&&&MapRange`
   --> src\python_process_info.rs:166:70
    |
166 |             if let Some(libpython) = libmaps.iter().min_by_key(|m| m.offset) {
    |                                                                      ^^^^^^ unknown field

[yinghai]

@fleimgruber yeah sorry probably I only covered the linux case.

[yinghai]

Pushed a change to keep the behavior unchanged for windows.

[fleimgruber]

@yinghai thanks for the quick change, py-spy compiles fine now, but I still get "Error: Failed to find python version from target process" on Python 3.10.11

[yinghai]

If it's windows, then I don't know how to solve it but maybe it's in the same line of idea.

[fleimgruber]

@benfred The addressed issues are still unresolved for Windows - should we create a tracking issue for a similar fix for Windows?