psutil [MacOS M1] Can not install via pip

Summary

OS: MacOS 12.5
Architecture: arm64
Psutil version: 5.9.1
Python version: 3.9.10
Type: installation

Description

bin/pip install psutil fails:

Collecting psutil
  Using cached psutil-5.9.1.tar.gz (479 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Building wheels for collected packages: psutil
  Building wheel for psutil (PEP 517) ... error
  ERROR: Command errored out with exit status 1:
…
  In file included from psutil/_psutil_osx.c:31:
  In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/IOKit.framework/Headers/IOKitLib.h:49:
  /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/IOKit.framework/Headers/IOTypes.h:81:49: error: expected ';' after top level declarator
  typedef mach_vm_address_t       IOVirtualAddress __kernel_ptr_semantics;
                                                  ^
  In file included from psutil/_psutil_osx.c:31:
  In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/IOKit.framework/Headers/IOKitLib.h:52:
  /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/IOKit.framework/Headers/OSMessageNotification.h:120:53: error: expected ';' after top level declarator
  typedef natural_t OSAsyncReference[kOSAsyncRefCount] __kernel_ptr_semantics;
                                                      ^
  psutil/_psutil_osx.c:102:17: warning: format specifies type 'long' but the argument has type 'pid_t' (aka 'int') [-Wformat]
                  pid, err, errno, mach_error_string(err));
                  ^~~
  psutil/_psutil_common.h:111:21: note: expanded from macro 'psutil_debug'
      fprintf(stderr, __VA_ARGS__); \
                      ^~~~~~~~~~~
  200 warnings and 2 errors generated.
  error: command '/usr/bin/clang' failed with exit code 1
  ----------------------------------------
  ERROR: Failed building wheel for psutil
Failed to build psutil
ERROR: Could not build wheels for psutil which use PEP 517 and cannot be installed directly

Aug 11 '22 13:08 sweh

I also get an error in _psutil_posix.c: psutil/_psutil_posix.c:514:21: error: duplicate case value '16' case(IFM_1000_T): ^ psutil/_psutil_posix.c:508:21: note: previous case defined here case(IFM_1000_TX): ^

Aug 11 '22 19:08 wtwarner

We're also getting a similar error - (IOKit/IOKitLib.h file not found)

Running with macOS version 12.5.1 (Apple M1)

  • Installing psutil (5.9.1): Failed

  EnvCommandError

  Command ['/Users/happy/Library/Caches/pypoetry/virtualenvs/server-SivvDSJ0-py3.9/bin/pip', 'install', '--no-deps', 'file:///Users/happy/Library/Caches/pypoetry/artifacts/ea/f1/eb/b46bae20206f1195d2d5e2584957896c44df5ad31dda53841e6ea8fd7e/psutil-5.9.1.tar.gz'] errored with the following return code 1, and output: 
  Processing /Users/happy/Library/Caches/pypoetry/artifacts/ea/f1/eb/b46bae20206f1195d2d5e2584957896c44df5ad31dda53841e6ea8fd7e/psutil-5.9.1.tar.gz
    Installing build dependencies: started
    Installing build dependencies: finished with status 'done'
    Getting requirements to build wheel: started
    Getting requirements to build wheel: finished with status 'done'
    Preparing metadata (pyproject.toml): started
    Preparing metadata (pyproject.toml): finished with status 'done'
  Building wheels for collected packages: psutil
    Building wheel for psutil (pyproject.toml): started
    Building wheel for psutil (pyproject.toml): finished with status 'error'
    ERROR: Command errored out with exit status 1:
     command: /Users/happy/Library/Caches/pypoetry/virtualenvs/server-SivvDSJ0-py3.9/bin/python /Users/happy/Library/Caches/pypoetry/virtualenvs/server-SivvDSJ0-py3.9/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py build_wheel /tmp/tmpxdd56_7q
         cwd: /private/tmp/pip-req-build-c57iv8zl
    Complete output (51 lines):
    running bdist_wheel
    running build
    running build_py
    creating build
    creating build/lib.macosx-10.6-arm64-cpython-39
    creating build/lib.macosx-10.6-arm64-cpython-39/psutil
    copying psutil/_pswindows.py -> build/lib.macosx-10.6-arm64-cpython-39/psutil
    copying psutil/_common.py -> build/lib.macosx-10.6-arm64-cpython-39/psutil
    copying psutil/__init__.py -> build/lib.macosx-10.6-arm64-cpython-39/psutil
    copying psutil/_psosx.py -> build/lib.macosx-10.6-arm64-cpython-39/psutil
    copying psutil/_psbsd.py -> build/lib.macosx-10.6-arm64-cpython-39/psutil
    copying psutil/_psaix.py -> build/lib.macosx-10.6-arm64-cpython-39/psutil
    copying psutil/_pslinux.py -> build/lib.macosx-10.6-arm64-cpython-39/psutil
    copying psutil/_compat.py -> build/lib.macosx-10.6-arm64-cpython-39/psutil
    copying psutil/_psposix.py -> build/lib.macosx-10.6-arm64-cpython-39/psutil
    copying psutil/_pssunos.py -> build/lib.macosx-10.6-arm64-cpython-39/psutil
    creating build/lib.macosx-10.6-arm64-cpython-39/psutil/tests
    copying psutil/tests/test_contracts.py -> build/lib.macosx-10.6-arm64-cpython-39/psutil/tests
    copying psutil/tests/test_connections.py -> build/lib.macosx-10.6-arm64-cpython-39/psutil/tests
    copying psutil/tests/runner.py -> build/lib.macosx-10.6-arm64-cpython-39/psutil/tests
    copying psutil/tests/test_unicode.py -> build/lib.macosx-10.6-arm64-cpython-39/psutil/tests
    copying psutil/tests/test_misc.py -> build/lib.macosx-10.6-arm64-cpython-39/psutil/tests
    copying psutil/tests/test_posix.py -> build/lib.macosx-10.6-arm64-cpython-39/psutil/tests
    copying psutil/tests/test_linux.py -> build/lib.macosx-10.6-arm64-cpython-39/psutil/tests
    copying psutil/tests/test_sunos.py -> build/lib.macosx-10.6-arm64-cpython-39/psutil/tests
    copying psutil/tests/__init__.py -> build/lib.macosx-10.6-arm64-cpython-39/psutil/tests
    copying psutil/tests/test_aix.py -> build/lib.macosx-10.6-arm64-cpython-39/psutil/tests
    copying psutil/tests/test_process.py -> build/lib.macosx-10.6-arm64-cpython-39/psutil/tests
    copying psutil/tests/test_bsd.py -> build/lib.macosx-10.6-arm64-cpython-39/psutil/tests
    copying psutil/tests/test_system.py -> build/lib.macosx-10.6-arm64-cpython-39/psutil/tests
    copying psutil/tests/test_osx.py -> build/lib.macosx-10.6-arm64-cpython-39/psutil/tests
    copying psutil/tests/test_memleaks.py -> build/lib.macosx-10.6-arm64-cpython-39/psutil/tests
    copying psutil/tests/test_windows.py -> build/lib.macosx-10.6-arm64-cpython-39/psutil/tests
    copying psutil/tests/__main__.py -> build/lib.macosx-10.6-arm64-cpython-39/psutil/tests
    copying psutil/tests/foo.py -> build/lib.macosx-10.6-arm64-cpython-39/psutil/tests
    copying psutil/tests/test_testutils.py -> build/lib.macosx-10.6-arm64-cpython-39/psutil/tests
    running build_ext
    building 'psutil._psutil_osx' extension
    creating build/temp.macosx-10.6-arm64-cpython-39
    creating build/temp.macosx-10.6-arm64-cpython-39/psutil
    creating build/temp.macosx-10.6-arm64-cpython-39/psutil/arch
    creating build/temp.macosx-10.6-arm64-cpython-39/psutil/arch/osx
    clang -fno-strict-aliasing -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -DPSUTIL_POSIX=1 -DPSUTIL_SIZEOF_PID_T=4 -DPSUTIL_VERSION=591 -DPSUTIL_OSX=1 -I/Users/happy/Library/Caches/pypoetry/virtualenvs/server-SivvDSJ0-py3.9/include -I/nix/store/wcmyw7rvadybk5cdcdd7xghwbzk2p44w-python3-3.9.9/include/python3.9 -c psutil/_psutil_common.c -o build/temp.macosx-10.6-arm64-cpython-39/psutil/_psutil_common.o
    clang-11: warning: argument unused during compilation: '-fno-strict-overflow' [-Wunused-command-line-argument]
    clang -fno-strict-aliasing -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -DPSUTIL_POSIX=1 -DPSUTIL_SIZEOF_PID_T=4 -DPSUTIL_VERSION=591 -DPSUTIL_OSX=1 -I/Users/happy/Library/Caches/pypoetry/virtualenvs/server-SivvDSJ0-py3.9/include -I/nix/store/wcmyw7rvadybk5cdcdd7xghwbzk2p44w-python3-3.9.9/include/python3.9 -c psutil/_psutil_osx.c -o build/temp.macosx-10.6-arm64-cpython-39/psutil/_psutil_osx.o
    clang-11: warning: argument unused during compilation: '-fno-strict-overflow' [-Wunused-command-line-argument]
    psutil/_psutil_osx.c:31:10: fatal error: 'IOKit/IOKitLib.h' file not found
    #include <IOKit/IOKitLib.h>
             ^~~~~~~~~~~~~~~~~~
    1 error generated.
    error: command '/nix/store/hcgsj2j0a49kx0k1jrs7b55rb9yhwwil-clang-wrapper-11.1.0/bin/clang' failed with exit code 1
    ----------------------------------------
    ERROR: Failed building wheel for psutil
  Failed to build psutil

Aug 19 '22 13:08 nw-ct

Similar issue with my new M1 MacBook

Collecting psutil==5.9.1
  Downloading psutil-5.9.1.tar.gz (479 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 479.1/479.1 kB 6.3 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: psutil
  Building wheel for psutil (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Building wheel for psutil (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [45 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.macosx-10.9-universal2-cpython-39
      creating build/lib.macosx-10.9-universal2-cpython-39/psutil
      copying psutil/_pswindows.py -> build/lib.macosx-10.9-universal2-cpython-39/psutil
      copying psutil/_common.py -> build/lib.macosx-10.9-universal2-cpython-39/psutil
      copying psutil/__init__.py -> build/lib.macosx-10.9-universal2-cpython-39/psutil
      copying psutil/_psosx.py -> build/lib.macosx-10.9-universal2-cpython-39/psutil
      copying psutil/_psbsd.py -> build/lib.macosx-10.9-universal2-cpython-39/psutil
      copying psutil/_psaix.py -> build/lib.macosx-10.9-universal2-cpython-39/psutil
      copying psutil/_pslinux.py -> build/lib.macosx-10.9-universal2-cpython-39/psutil
      copying psutil/_compat.py -> build/lib.macosx-10.9-universal2-cpython-39/psutil
      copying psutil/_psposix.py -> build/lib.macosx-10.9-universal2-cpython-39/psutil
      copying psutil/_pssunos.py -> build/lib.macosx-10.9-universal2-cpython-39/psutil
      creating build/lib.macosx-10.9-universal2-cpython-39/psutil/tests
      copying psutil/tests/test_contracts.py -> build/lib.macosx-10.9-universal2-cpython-39/psutil/tests
      copying psutil/tests/test_connections.py -> build/lib.macosx-10.9-universal2-cpython-39/psutil/tests
      copying psutil/tests/runner.py -> build/lib.macosx-10.9-universal2-cpython-39/psutil/tests
      copying psutil/tests/test_unicode.py -> build/lib.macosx-10.9-universal2-cpython-39/psutil/tests
      copying psutil/tests/test_misc.py -> build/lib.macosx-10.9-universal2-cpython-39/psutil/tests
      copying psutil/tests/test_posix.py -> build/lib.macosx-10.9-universal2-cpython-39/psutil/tests
      copying psutil/tests/test_linux.py -> build/lib.macosx-10.9-universal2-cpython-39/psutil/tests
      copying psutil/tests/test_sunos.py -> build/lib.macosx-10.9-universal2-cpython-39/psutil/tests
      copying psutil/tests/__init__.py -> build/lib.macosx-10.9-universal2-cpython-39/psutil/tests
      copying psutil/tests/test_aix.py -> build/lib.macosx-10.9-universal2-cpython-39/psutil/tests
      copying psutil/tests/test_process.py -> build/lib.macosx-10.9-universal2-cpython-39/psutil/tests
      copying psutil/tests/test_bsd.py -> build/lib.macosx-10.9-universal2-cpython-39/psutil/tests
      copying psutil/tests/test_system.py -> build/lib.macosx-10.9-universal2-cpython-39/psutil/tests
      copying psutil/tests/test_osx.py -> build/lib.macosx-10.9-universal2-cpython-39/psutil/tests
      copying psutil/tests/test_memleaks.py -> build/lib.macosx-10.9-universal2-cpython-39/psutil/tests
      copying psutil/tests/test_windows.py -> build/lib.macosx-10.9-universal2-cpython-39/psutil/tests
      copying psutil/tests/__main__.py -> build/lib.macosx-10.9-universal2-cpython-39/psutil/tests
      copying psutil/tests/foo.py -> build/lib.macosx-10.9-universal2-cpython-39/psutil/tests
      copying psutil/tests/test_testutils.py -> build/lib.macosx-10.9-universal2-cpython-39/psutil/tests
      running build_ext
      building 'psutil._psutil_osx' extension
      creating build/temp.macosx-10.9-universal2-cpython-39
      creating build/temp.macosx-10.9-universal2-cpython-39/psutil
      creating build/temp.macosx-10.9-universal2-cpython-39/psutil/arch
      creating build/temp.macosx-10.9-universal2-cpython-39/psutil/arch/osx
      clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders -iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/Headers -arch x86_64 -Werror=implicit-function-declaration -I/opt/homebrew/opt/openblas/include -DPSUTIL_POSIX=1 -DPSUTIL_SIZEOF_PID_T=4 -DPSUTIL_VERSION=591 -DPSUTIL_OSX=1 -I/Users/tony/repos/py-compute-poc/venv/include -I/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/Headers -c psutil/_psutil_common.c -o build/temp.macosx-10.9-universal2-cpython-39/psutil/_psutil_common.o
      Executable requires at least macOS 12.0, but is being run on macOS 10.16, and so is exiting.
      error: command '/usr/bin/clang' failed with exit code 1
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for psutil
Failed to build psutil
ERROR: Could not build wheels for psutil, which is required to install pyproject.toml-based projects

Oct 06 '22 18:10 tonycombocurve

@nw-ct You seem to use clang provided by Nix, I'd use the one from Apple's command line tools by default. I guess the Nix one cannot find the macOS SDK, at least with default configuration. If you haven't installed them yet, you can do so with xcode-select --install. Make sure clang is invoked from /usr/bin/clang and it is the Apple-provided one.

@tonycombocurve Do you have something like SYSTEM_VERSION_COMPAT set in your environment variables? IIRC when set it fakes a 10.x-compatible version (the non-existing 10.16) to be compatible with old programs checking the OS version with a 10.x pattern. Check your .bashrc/.zshrc/whatever. It's not really something you want to have configured globally, but rather for specific use-cases only.

I cannot tell it will solve both your problems (no idea if anyone succeeded an M1 install yet), but it will get you further, maybe you'll get something similar to the OP at least :P Could be useful for everyone to also check for Command Line Tools updates in the software update settings menu.

No Apple Silicon machine here to test myself...

Oct 08 '22 09:10 ben9923

@ben9923 thanks for following up. i believe that i discovered the issue over the weekend. it had to do with xcode not being initialized. this small step ended up resolving the issue: xcodebuild -runFirstLaunch

Oct 10 '22 21:10 tonycombocurve

As a result of merging https://github.com/giampaolo/psutil/pull/2040 and releasing MacOS M1 packages, this is now solved for me. Thank you!

Collecting psutil
  Downloading psutil-5.9.3-cp39-cp39-macosx_11_0_arm64.whl (243 kB)
     |████████████████████████████████| 243 kB 1.9 MB/s
Installing collected packages: psutil
Successfully installed psutil-5.9.3

Oct 19 '22 05:10 sweh

psutil psutil copied to clipboard

[MacOS M1] Can not install via pip

Summary

Description

psutil
psutil copied to clipboard