Failed to build libproc bindings

[micouy]

System: macOS 11.0.1

Cargo.toml:

[package]
# ...

[dependencies]
libproc = "0.12.0"

Error:

   Compiling libproc v0.12.0
error: failed to run custom build command for `libproc v0.12.0`

Caused by:
  process didn't exit successfully: `<PROJECT PATH>/target/debug/build/libproc-7ab682cfcd895988/build-script-build` (exit status: 101)
  --- stderr
  <HOME>/.cargo/registry/src/github.com-1ecc6299db9ec823/libproc-0.12.0/libproc_rs.h:1:10: fatal error: 'libproc.h' file not found
  <HOME>/.cargo/registry/src/github.com-1ecc6299db9ec823/libproc-0.12.0/libproc_rs.h:1:10: fatal error: 'libproc.h' file not found, err: true
  thread 'main' panicked at 'Failed to build libproc bindings: ()', <HOME>/.cargo/registry/src/github.com-1ecc6299db9ec823/libproc-0.12.0/build.rs:9:10
  stack backtrace:
     0: rust_begin_unwind
               at /rustc/62b272d25c5bb8b6bb8ac73797d82b8b9a1eabda/library/std/src/panicking.rs:584:5
     1: core::panicking::panic_fmt
               at /rustc/62b272d25c5bb8b6bb8ac73797d82b8b9a1eabda/library/core/src/panicking.rs:142:14
     2: core::result::unwrap_failed
               at /rustc/62b272d25c5bb8b6bb8ac73797d82b8b9a1eabda/library/core/src/result.rs:1805:5
     3: core::result::Result<T,E>::expect
               at /rustc/62b272d25c5bb8b6bb8ac73797d82b8b9a1eabda/library/core/src/result.rs:1055:23
     4: build_script_build::main
               at ./build.rs:6:20
     5: core::ops::function::FnOnce::call_once
               at /rustc/62b272d25c5bb8b6bb8ac73797d82b8b9a1eabda/library/core/src/ops/function.rs:248:5
  note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

If I can give you any more info, please notify me in the comment.

[andrewdavidmackenzie]

Thank for the report. Everything seem to have fallen apart on multiple projects over last two days. Unfortunately, I have just started a week's vacation away from the computer.

If there was a working PR I could merge it from mobile or iPad while away, but might struggle to publish to crates.io, unless maybe from CodeSpaces.....

[micouy]

Thanks for the quick response. Don't worry, perhaps there's a problem on my side. Happy vacation, don't waste it on GitHub!

[andrewdavidmackenzie]

It's compiling fine for me on my local machine (and nightly CI) at my first attempt. (macos 12.4).

It seems to be not finding the headers ('libproc.h' file not found), which is maybe related to use of pkgconfig to find them?

Was this building for you before and it broke, or this is preventing you getting a local build working for the first time?

[micouy]

This was a fresh build.

[andrewdavidmackenzie]

I'm back from leave and updating this lib. I have merged significant contributions and updated version to 0.13.0 and tagged and published to crates.io. Can you update and test again and let me know (or close the issue if not). Thanks.

[sporkmonger]

For what it's worth, I'm encountering the same issue on the latest version using Monterey 12.6.2 on an Intel machine.

[andrewdavidmackenzie]

Nightly CI check (which includes macos11 and 12) is still passing fine. https://github.com/andrewdavidmackenzie/libproc-rs/actions/runs/5659636456

I am on Ventura, so not able to check on Monterey.

[jrabinow]

I'm on Ventura as well, but I'm getting the same build issue:

$ make
cargo clippy --all --tests --all-targets --all-features -- -D warnings
   Compiling libproc v0.14.0 (/private/tmp/libproc-rs)
error: failed to run custom build command for `libproc v0.14.0 (/private/tmp/libproc-rs)`

Caused by:
  process didn't exit successfully: `/private/tmp/libproc-rs/target/debug/build/libproc-3d528c68cc69b4fd/build-script-build` (exit status: 101)
  --- stderr
  /private/tmp/libproc-rs/libproc_rs.h:1:10: fatal error: 'libproc.h' file not found
  thread 'main' panicked at 'Failed to build libproc bindings: ClangDiagnostic("/private/tmp/libproc-rs/libproc_rs.h:1:10: fatal error: 'libproc.h' file not found\n")', build.rs:11:10
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
make: *** [Makefile:6: clippy] Error 101

macOS 13.4.1 22F82 arm64
Xcode 14.3 14E222b

The project builds successfully with the following patch:

diff --git a/build.rs b/build.rs
index 9f687e30..a81ca3a3 100644
--- a/build.rs
+++ b/build.rs
@@ -6,7 +6,7 @@ fn main() {
     let bindings = bindgen::builder()
         .header_contents("libproc_rs.h", "#include <libproc.h>")
         .layout_tests(false)
-        .clang_args(&["-x", "c++"])
+        .clang_args(&["-x", "c++", "-I", "/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/"])
         .generate()
         .expect("Failed to build libproc bindings");

I'm not sure applying this patch verbatim is a good idea though, unless the dependency on command-line tools is made explicit

[andrewdavidmackenzie]

working fine for me... something to do with bindgen?

bindgen --version
bindgen 0.66.1

What do you have?

[andrewdavidmackenzie]

That patch would be fine for me, as libproc.h in in that dir.

want to send it as a PR?

But I'd like to understand what's up and why build finds it on my system and not on yours....

Are both the failing machines Arm (M*) architecture? I'm on Intel still...

[jrabinow]

I don't have bindgen installed. I just installed cbindgen from macports but it seems it's a different project

$ cbindgen --version
cbindgen 0.24.6

My machine is an ARM M1 chip indeed, I think that @sporkmonger reported his was intel-based

[andrewdavidmackenzie]

Want to send a PR and others could see if your branch works for them, prior to merging?

[jrabinow]

PR is up. I'm wondering though, where is libproc.h located on your filesystem and why is it found on yours and not on mine? I glanced through https://stackoverflow.com/q/19852199 but didn't see anything obvious, you mentioned pkg-config above which might also play a role.

[andrewdavidmackenzie]

Location of libproc.h

It's in lots of places :-) including the same as yours:

find / -name libproc.h 2>&1 |grep -v Permission
/Library/Developer/CommandLineTools/SDKs/MacOSX13.3.sdk/usr/include/libproc.h
/Library/Developer/CommandLineTools/SDKs/MacOSX13.1.sdk/usr/include/libproc.h
/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/include/libproc.h
/Library/Developer/CommandLineTools/SDKs/MacOSX12.1.sdk/usr/include/libproc.h
/Library/Developer/CommandLineTools/SDKs/MacOSX12.3.sdk/usr/include/libproc.h
/System/Volumes/Data/Library/Developer/CommandLineTools/SDKs/MacOSX13.3.sdk/usr/include/libproc.h
/System/Volumes/Data/Library/Developer/CommandLineTools/SDKs/MacOSX13.1.sdk/usr/include/libproc.h
/System/Volumes/Data/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/include/libproc.h
/System/Volumes/Data/Library/Developer/CommandLineTools/SDKs/MacOSX12.1.sdk/usr/include/libproc.h
/System/Volumes/Data/Library/Developer/CommandLineTools/SDKs/MacOSX12.3.sdk/usr/include/libproc.h
/System/Volumes/Data/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/libproc.h
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/libproc.h

The trick will be to figure out which one my build is using, and the differences with your install. If you can see which of those you don't have installed, it's probably one of those used by default.

I notice that MacOSX.sdk is a link to a specific (latest?) version of the SDK, thus:

ll /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk             
lrwxr-xr-x  1 root  wheel  14  6 Jul 13:37 /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -> MacOSX13.3.sdk

Is that the case for you?

ll /Library/Developer/CommandLineTools/SDKs           
total 0
drwxr-xr-x  11 root  wheel  352  6 Jul 13:37 .
drwxrwxr-x   5 root  wheel  160  6 Jul 13:37 ..
lrwxr-xr-x   1 root  wheel   14  6 Jul 13:37 MacOSX.sdk -> MacOSX13.3.sdk
drwxr-xr-x   7 root  wheel  224 26 Mar  2020 MacOSX10.14.sdk
drwxr-xr-x   3 root  wheel   96 30 Oct  2021 MacOSX10.15.sdk
drwxr-xr-x   7 root  wheel  224 31 Oct  2022 MacOSX12.1.sdk
drwxr-xr-x   7 root  wheel  224  6 Jul 13:37 MacOSX12.3.sdk
lrwxr-xr-x   1 root  wheel   14  6 Jul 13:35 MacOSX12.sdk -> MacOSX12.3.sdk
drwxr-xr-x   7 root  wheel  224 12 Nov  2022 MacOSX13.1.sdk
drwxr-xr-x   7 root  wheel  224 10 Mar 01:58 MacOSX13.3.sdk
lrwxr-xr-x   1 root  wheel   14  6 Jul 13:34 MacOSX13.sdk -> MacOSX13.3.sdk

pkg-config

I wondered about pkg-config, but I think that this (libproc.h) being part of the OS and not an installed package that it won't apply.

`pkg-config --list-all|grep libproc``

doesn't have anything, so I think that's not the case.

The fix

Given it seems that xcode links that generic/latest SDK dir into one with the header file, and I have it also, I assume your fix in the PR is able to be used by all - and we could go with it.

[andrewdavidmackenzie]

Fixed by https://github.com/andrewdavidmackenzie/libproc-rs/pull/103

[jrabinow]

Sorry for the late reply.

I notice that MacOSX.sdk is a link to a specific (latest?) version of the SDK, [...] Is that the case for you?

Yes, I have the same

Given it seems that xcode links that generic/latest SDK dir into one with the header file, and I have it also, I assume your fix in the PR is able to be used by all - and we could go with it.

Just an FYI, there's a relevant envvar here for SDKs: SDKROOT. I'm not sure if it applies for command-line tools or xcode but if any problems happen in the future, it might be worth taking a look at the environment.

Also this looks like it might be relevant: https://developer.apple.com/library/archive/technotes/tn2339/_index.html Of note, the docs say xcode is required to be installed for command-line tools. I'm not sure that's the case: this may still be a valid workaround.

I quite dislike the fact we're adding this dependency on commandline tools to the build process, but I'm not knowledgeable enough to know how to bypass it. It might be worth making the dependency explicit in the README though.

Last thing, I'm wondering if the pattern here is with the compiler suite.

$ which -a clang
/opt/local/bin/clang
/opt/local/libexec/llvm/bin/clang
/usr/bin/clang
$ clang --version
clang version 16.0.6
Target: arm64-apple-darwin22.5.0
Thread model: posix
InstalledDir: /opt/local/libexec/llvm-16/bin
$ /usr/bin/clang --version
Apple clang version 14.0.3 (clang-1403.0.22.14.1)
Target: arm64-apple-darwin22.5.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

Assuming the compiler used here is clang: as you can see, my clang is not the system clang, it's one I installed via macports. Maybe the clean fix here would be to explicitly set the full compiler path? I'm afraid I don't have experience with the rust build system but if you sent me a patch, I could give it a shot.

[andrewdavidmackenzie]

Replying to your points:

• SDKROOT I don't have that set, so not sure who sets it. But it doesn't see "universal"?
• "Dependency on commandline tools", you mean clang and the SDK right?
  • I'm not sure you can avoid the SDK dependency to get libproc.h?
  • An alternative compiler on mac for mac libraries and FFI seems like a challenge...I mean gnu C/C++ is the main other option (or zig or something?)
• I am kind of assuming that anyone building on macos will be OK with those dependencies. Let's see!

Did the release / HEAD of main with the change build OK for you now?

[jrabinow]

SDKROOT isn't defined on my system either. I'm probably being overly cautious mentioning it, I'm afraid I don't understand apple's build system very well. Here is what I do know: the /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk symlink we were talking about above seems to be managed by the system, and it's not impossible for it to be defined according to the SDKROOT envvar using some apple magic behind the scenes. In case there's a bugreport down the road involving libproc.h, this might be something to look at.

"Dependency on commandline tools", you mean clang and the SDK right?

Specifically on the apple commandline tools package I'm not sure if clang comes installed by default or not on mac, but libproc.h does seem to come with the SDK, which I think is only present on the system if commandline tools is installed. If the commandline tools package not installed but xcode is installed, it might be possible to reference another directory under /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs

Re clang: sorry I was unclear. Since libproc-rs builds fine on your system but not on mine, I thought that my clang (installed from macports) might be isolated from the rest of the OS as is par for the course for macports. I ran a test though and that's not what the root cause is.

Did the release / HEAD of main with the change build OK for you now?

Yes, now it goes through no probs, thanks!

[jrabinow]

@andrewdavidmackenzie now that https://github.com/andrewdavidmackenzie/libproc-rs/pull/103 has been merged in, would it make sense to create a new release to unblock https://github.com/benfred/py-spy/issues/596 ?

[andrewdavidmackenzie]

"0.14.2" published on crates.io (patch release as no functional change)

Tag/Release v0.14.2 (see https://github.com/andrewdavidmackenzie/libproc-rs/releases/tag/v0.14.2)

[andrewdavidmackenzie]

If that works for you, either close the issue, or let me know and I'll close it.

[jrabinow]

I can't close the issue, but that should definitely do the trick! Thank you!

[andrewdavidmackenzie]

Ah yes, closed when the PR was made - sorry. Glad that worked for you.