Python package install fails in nix due to using C compiler

[matthewd-canva]

Hi,

Attempting to install py-tlsh in my environment produces an error, due to setuptools choosing a C instead of C++ compiler.

I'm guessing this has something to do with the setuptools version not auto detecting C++ - maybe you could explicitly set it?

Setup

I'm using the following environment on an M1 Mac (aarch64):

shell.nix

{ pkgs ? import <nixpkgs> {} }:
pkgs.mkShell {
  buildInputs = with pkgs.buildPackages; [
    python310
  ];
  shellHook = ''
    python -m venv .venv
    . .venv/bin/activate
  '';
}

Strangely the install works as expected when using host Python, just not in the nix shell. Running pip debug --verbose in the two environments is identical, excluding the build dates of Python itself. Also install works as expected when using the same nix shell inside an Ubuntu docker container on the same host.

What happens

$ pip install py-tlsh
Collecting py-tlsh
  Using cached py-tlsh-4.7.2.tar.gz (42 kB)
  Preparing metadata (setup.py) ... done
Using legacy 'setup.py install' for py-tlsh, since package 'wheel' is not installed.
Installing collected packages: py-tlsh
  Running setup.py install for py-tlsh ... error
  error: subprocess-exited-with-error
  
  × Running setup.py install for py-tlsh did not run successfully.
  │ exit code: 1
  ╰─> [18 lines of output]
      running install
      /tmp/py-tlsh/.venv/lib/python3.10/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
        warnings.warn(
      running build
      running build_ext
      building 'tlsh' extension
      creating build
      creating build/temp.macosx-11.0-arm64-cpython-310
      creating build/temp.macosx-11.0-arm64-cpython-310/src
      clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Iinclude -I/tmp/py-tlsh/.venv/include -I/nix/store/qsp3vpszsab7zn4h364sfkvnp2qilrrl-python3-3.10.6/include/python3.10 -c src/tlsh.cpp -o build/temp.macosx-11.0-arm64-cpython-310/src/tlsh.o -DBUCKETS_128
      clang-11: warning: argument unused during compilation: '-fno-strict-overflow' [-Wunused-command-line-argument]
      clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Iinclude -I/tmp/py-tlsh/.venv/include -I/nix/store/qsp3vpszsab7zn4h364sfkvnp2qilrrl-python3-3.10.6/include/python3.10 -c src/tlsh_impl.cpp -o build/temp.macosx-11.0-arm64-cpython-310/src/tlsh_impl.o -DBUCKETS_128
      clang-11: warning: argument unused during compilation: '-fno-strict-overflow' [-Wunused-command-line-argument]
      src/tlsh_impl.cpp:62:10: fatal error: 'string' file not found
      #include <string>
               ^~~~~~~~
      1 error generated.
      error: command '/nix/store/b9bh3z78rqff14bf0fd78xb07b5cnwdw-clang-wrapper-11.1.0/bin/clang' failed with exit code 1
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> py-tlsh

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.

Workaround

This should work without forcing CC environment var.

$ CC=clang++ pip install py-tlsh
Collecting py-tlsh
  Using cached py-tlsh-4.7.2.tar.gz (42 kB)
  Preparing metadata (setup.py) ... done
Using legacy 'setup.py install' for py-tlsh, since package 'wheel' is not installed.
Installing collected packages: py-tlsh
  Running setup.py install for py-tlsh ... done
Successfully installed py-tlsh-4.7.2

[jonjoliver]

matthewd: should I change the #include to #include <string.h>

That might help if a range of situations

[matthewd-canva]

The file contains C++ so the include is correct. But note that nothing from that include is used in that file. Also note that string.h is included, but that is incorrect for C++ (it should be cstring, same case for errno.h -> cerrno.h).

It's possible that you could remove the string include and it would work, but that might just mask the error - is clang compiling C++ correctly or is it trying to compile C++ as C?