py-spy icon indicating copy to clipboard operation
py-spy copied to clipboard

Error when profiling with --native flag

Open a-wang24 opened this issue 5 years ago • 1 comments

Hello,

First, thank you for this tool! It's been invaluable for profiling Python, Cython, and C++ extensions together.

I am encountering an error I can't quite figure out when using py-spy record with the same command also spawning the Python process. Anytime I add the --native flag, py-spy aborts with a core dump, but the Python process continues running as expected (i.e. I continue seeing the Python process's output in stdout after py-spy aborts). Even with RUST_BACKTRACE=full set, I still only see Aborted (core dumped). Some dummy example commands below.

$ py-spy record -o flamegraph.svg -- python my_script.py <script args>  // Works great
$ py-spy record -n -o flamegraph.svg -- python my_script.py <script args>  // Fails with just Aborted (core dumped)

If I separately run the python script and run py-spy record -n -o graph.svg --pid <script pid>, everything works great. Any idea what's going on here? or ideas for how to diagnose? I'm admittedly a little in over my head here.

System:

  • Docker container with --cap-add SYS_PTRACE
  • OS: Ubuntu 18.04
  • Python 3.6
  • py-spy 0.3.3 installed via pip

a-wang24 avatar Dec 31 '20 22:12 a-wang24

I haven't seen a segfault here before =( I'm guessing it's probably segfaulting in libunwind trying to get the native trace from your program but can't really be sure - and have no idea why it is segfaulting

Can you try getting the stack trace from py-spy where it fails?

Running py-spy under gdb should do this gdb --args py-spy record -n -o flamegraph.svg -- python myscript.py

Also setting RUST_LOG=info (or even RUST_LOG=debug) will spit out some more diagnostic information that might help narrow this down.

benfred avatar Jan 06 '21 05:01 benfred