pprof-rs icon indicating copy to clipboard operation
pprof-rs copied to clipboard

Trace/BPT trap on Mac

Open DavidBJaffe opened this issue 2 years ago • 6 comments

This code:

use pprof::ProfilerGuard;
static mut GUARD: Option<ProfilerGuard<'static>> = None;
static mut BLACKLIST: Vec<String> = Vec::new();
pub fn start_profilingx(blacklist: &[String]) {
    let frequency = 1000;
    unsafe {
        BLACKLIST = blacklist.to_vec();
        GUARD = Some(pprof::ProfilerGuard::new(frequency).unwrap());
    }
}
pub fn profiling_blacklistx() -> Vec<String> {
    vec!["woof".to_string()]
}
fn main() {
    start_profilingx(&profiling_blacklistx());
    let mut mc = 0;
    for i in 0..10_000 {
        for j in 0..10_000 {
            let n = i * j;
            mc = (mc + n) % 999;
        }
    }
    eprintln!("mc = {}", mc);
}

yields Trace/BPT trap when I run it, at least most of the time. Other similar code does this most of the time.

It is exceptionally finicky. For example, changing vec!["woof".to_string()] to vec![] eliminates the problem.

This is using master on pprof, but also occurs in 0.6.

I believe that the problem may be restricted to particular Mac versions. I have:

rustc 1.60.0 OSX 12.4 chip = apple M1 Pro

[profile.dev] debug = 1 opt-level = 3 split-debuginfo = "unpacked"

DavidBJaffe avatar May 30 '22 14:05 DavidBJaffe

I have now encountered the same error when not using pprof at all.

DavidBJaffe avatar May 30 '22 17:05 DavidBJaffe

Reopening. The problem is presumably with libunwind, but it makes pprof unusable on a Mac, or more precisely on my Mac. Another person reproduced the problem on a non-M1 Mac. Maybe the problem occurs on all Macs.

DavidBJaffe avatar May 30 '22 18:05 DavidBJaffe

I'm having this problem as well, on a Mac M1 pro, same thing as yours, no profile.dev settings. Throws somewhere on libunwind when calling .pprof() at this part:

libunwind::UnwindCursor<libunwind::LocalAddressSpace, libunwind::Registers_arm64>::step() (@libunwind::UnwindCursor<libunwind::LocalAddressSpace, libunwind::Registers_arm64>::step():162)

ricardopieper avatar Jun 02 '22 17:06 ricardopieper

I've actually been unable to get any form of profiling to work on my Mac, so for me at least, getting this to work would be phenomenally helpful.

DavidBJaffe avatar Jun 03 '22 15:06 DavidBJaffe

For mac user, the frame-pointer is a more suggested way to get the backtrace, as the mac os has established an ecosystem of using the frame pointer, which is much more efficient and simpler than dwarf. More information can be found in https://github.com/tikv/pprof-rs/issues/75#issuecomment-1105687213 .

In the latest nightly of pprof-rs, enabling the frame-pointer feature will use the frame-pointer to get the backtrace. For the 0.9 release, you could disable the backtrace-rs feature (which is the default) and enable the frame-pointer feature.

If you insist on using the dwarf way, you could try to add more things to the blacklist (according to the fault backtrace). It seems that some parts of mac os (context.pc) don't deliver with understandable dwarf information. We have experience the same situation on https://github.com/tikv/tikv/issues/9957#issuecomment-813368686 .

YangKeao avatar Jun 06 '22 03:06 YangKeao

Yes, that works really well, thank you!

DavidBJaffe avatar Dec 13 '22 15:12 DavidBJaffe