py-spy icon indicating copy to clipboard operation
py-spy copied to clipboard

Panic when profiling python process in container

Open YagelSheli opened this issue 5 years ago • 2 comments

After running Py-Spy in a container (with CAP_PTRACE) and running the following command py-spy record -r 100 -s -o /tmp/flame-graph.svg run_python_code_to_profile.sh The following error raised: thread ‘main’ panicked at ‘called Option::unwrap() on a None value’, /rustc/5e1a799842ba6ed4a57e91f7ab9435947482f7d8/src/libcore/macros/mod.rs:15:40

Backtrace and other information are available upon request.

YagelSheli avatar Oct 12 '20 08:10 YagelSheli

Can you provide a backtrace? It's hard to tell just looking at this what the error could be

benfred avatar Oct 30 '20 05:10 benfred

I was trying out py-spy on a containerised setup as well and managed to hit the same error. I then set the environment variable to get a backtrace and kept trying, and eventually got this backtrace:

Click to expand
root@container:~# time RUST_BACKTRACE=1 py-spy record -o flames.svg -d 10 -s --nonblocking -p 1
py-spy> Sampling process 100 times a second for 10 seconds. Press Control-C to exit.

^Cthread 'main' panicked at 'called `Option::unwrap()` on a `None` value', /rustc/5e1a799842ba6ed4a57e91f7ab9435947482f7d8/src/libcore/macros/mod.rs:15:40
stack backtrace:
   0: backtrace::backtrace::libunwind::trace
             at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.40/src/backtrace/libunwind.rs:88
   1: backtrace::backtrace::trace_unsynchronized
             at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.40/src/backtrace/mod.rs:66
   2: std::sys_common::backtrace::_print_fmt
             at src/libstd/sys_common/backtrace.rs:84
   3: <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt
             at src/libstd/sys_common/backtrace.rs:61
   4: core::fmt::write
             at src/libcore/fmt/mod.rs:1025
   5: std::io::Write::write_fmt
             at src/libstd/io/mod.rs:1426
   6: std::sys_common::backtrace::_print
             at src/libstd/sys_common/backtrace.rs:65
   7: std::sys_common::backtrace::print
             at src/libstd/sys_common/backtrace.rs:50
   8: std::panicking::default_hook::{{closure}}
             at src/libstd/panicking.rs:193
   9: std::panicking::default_hook
             at src/libstd/panicking.rs:210
  10: std::panicking::rust_panic_with_hook
             at src/libstd/panicking.rs:471
  11: rust_begin_unwind
             at src/libstd/panicking.rs:375
  12: core::panicking::panic_fmt
             at src/libcore/panicking.rs:84
  13: core::panicking::panic
             at src/libcore/panicking.rs:51
  14: py_spy::run_spy_command
  15: py_spy::main
  16: std::rt::lang_start::{{closure}}
  17: std::rt::lang_start_internal::{{closure}}
             at src/libstd/rt.rs:52
  18: std::panicking::try::do_call
             at src/libstd/panicking.rs:292
  19: __rust_maybe_catch_panic
             at src/libpanic_unwind/lib.rs:78
  20: std::panicking::try
             at src/libstd/panicking.rs:270
  21: std::panic::catch_unwind
             at src/libstd/panic.rs:394
  22: std::rt::lang_start_internal
             at src/libstd/rt.rs:51
  23: main
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
[2020-12-16T20:34:23.271689458Z ERROR py_spy::sampler] Failed to get process tree: poisoned lock: another task failed inside
[2020-12-16T20:34:23.271850124Z ERROR py_spy::sampler] Failed to get process tree: poisoned lock: another task failed inside
[2020-12-16T20:34:23.271955655Z ERROR py_spy::sampler] Failed to get process tree: poisoned lock: another task failed inside

(lines above repeated 1100 times)

I should probably mention that I pressed control+C while py-spy was running because I was afraid of the performance implications it would have, and especially since it was running for longer than the 10 seconds I had told it to sample for. I don't know if that impacts the output.

Tenzer avatar Dec 16 '20 20:12 Tenzer