--config-settings not passing --build-option values to clang on macos

Open jmkacz opened this issue 2 years ago • 1 comments

Description

pygraphviz cannot be installed using --config-settings and --build-option. You have to set LDFLAGS and CFLAGS environment variables.

Related to https://github.com/pygraphviz/pygraphviz/issues/398, but pip is not behaving as expected.

Expected behavior

pygraphviz is installable via the recommended process

pip version

24.0

Python version

3.12.0

OS

macOS Sonoma (14.1.2)

How to Reproduce

Environment

Details

(venv) jonathankaczynski@jkaczynski-mac ~ % sysctl -n machdep.cpu.brand_string
Apple M1 Pro
(venv) jonathankaczynski@jkaczynski-mac ~ % sw_vers -productVersion
14.1.2
jonathankaczynski@jkaczynski-mac ~ % pyenv versions
  system
  2.7.18
  3.6.15
  3.7.17
  3.10.13
  3.11.6
* 3.12.0 (set by PYENV_VERSION environment variable)
jonathankaczynski@jkaczynski-mac ~ % python --version
Python 3.12.0
jonathankaczynski@jkaczynski-mac ~ % python -m pip --version
pip 24.0 from /Users/jonathankaczynski/.pyenv/versions/3.12.0/lib/python3.12/site-packages/pip (python 3.12)

Setup

jonathankaczynski@jkaczynski-mac ~ % python -m venv /tmp/venv
jonathankaczynski@jkaczynski-mac ~ % source /tmp/venv/bin/activate

Attempt 1

Attempting python -m pip install pygraphviz fails, but that's expected as homebrew put graphviz files in a non-standard location.

Details

(venv) jonathankaczynski@jkaczynski-mac ~ % python -m pip install pygraphviz
Looking in indexes: https://pypi.org/simple
Collecting pygraphviz
  Using cached pygraphviz-1.12.tar.gz (104 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: pygraphviz
  Building wheel for pygraphviz (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for pygraphviz (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [56 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.macosx-14.1-arm64-cpython-312
      creating build/lib.macosx-14.1-arm64-cpython-312/pygraphviz
      copying pygraphviz/scraper.py -> build/lib.macosx-14.1-arm64-cpython-312/pygraphviz
      copying pygraphviz/graphviz.py -> build/lib.macosx-14.1-arm64-cpython-312/pygraphviz
      copying pygraphviz/__init__.py -> build/lib.macosx-14.1-arm64-cpython-312/pygraphviz
      copying pygraphviz/agraph.py -> build/lib.macosx-14.1-arm64-cpython-312/pygraphviz
      copying pygraphviz/testing.py -> build/lib.macosx-14.1-arm64-cpython-312/pygraphviz
      creating build/lib.macosx-14.1-arm64-cpython-312/pygraphviz/tests
      copying pygraphviz/tests/test_unicode.py -> build/lib.macosx-14.1-arm64-cpython-312/pygraphviz/tests
      copying pygraphviz/tests/test_scraper.py -> build/lib.macosx-14.1-arm64-cpython-312/pygraphviz/tests
      copying pygraphviz/tests/test_readwrite.py -> build/lib.macosx-14.1-arm64-cpython-312/pygraphviz/tests
      copying pygraphviz/tests/test_string.py -> build/lib.macosx-14.1-arm64-cpython-312/pygraphviz/tests
      copying pygraphviz/tests/__init__.py -> build/lib.macosx-14.1-arm64-cpython-312/pygraphviz/tests
      copying pygraphviz/tests/test_html.py -> build/lib.macosx-14.1-arm64-cpython-312/pygraphviz/tests
      copying pygraphviz/tests/test_node_attributes.py -> build/lib.macosx-14.1-arm64-cpython-312/pygraphviz/tests
      copying pygraphviz/tests/test_drawing.py -> build/lib.macosx-14.1-arm64-cpython-312/pygraphviz/tests
      copying pygraphviz/tests/test_repr_mimebundle.py -> build/lib.macosx-14.1-arm64-cpython-312/pygraphviz/tests
      copying pygraphviz/tests/test_subgraph.py -> build/lib.macosx-14.1-arm64-cpython-312/pygraphviz/tests
      copying pygraphviz/tests/test_close.py -> build/lib.macosx-14.1-arm64-cpython-312/pygraphviz/tests
      copying pygraphviz/tests/test_edge_attributes.py -> build/lib.macosx-14.1-arm64-cpython-312/pygraphviz/tests
      copying pygraphviz/tests/test_clear.py -> build/lib.macosx-14.1-arm64-cpython-312/pygraphviz/tests
      copying pygraphviz/tests/test_layout.py -> build/lib.macosx-14.1-arm64-cpython-312/pygraphviz/tests
      copying pygraphviz/tests/test_attribute_defaults.py -> build/lib.macosx-14.1-arm64-cpython-312/pygraphviz/tests
      copying pygraphviz/tests/test_graph.py -> build/lib.macosx-14.1-arm64-cpython-312/pygraphviz/tests
      running egg_info
      writing pygraphviz.egg-info/PKG-INFO
      writing dependency_links to pygraphviz.egg-info/dependency_links.txt
      writing top-level names to pygraphviz.egg-info/top_level.txt
      reading manifest file 'pygraphviz.egg-info/SOURCES.txt'
      reading manifest template 'MANIFEST.in'
      warning: no files found matching '*.png' under directory 'doc'
      warning: no files found matching '*.html' under directory 'doc'
      warning: no files found matching '*.txt' under directory 'doc'
      warning: no files found matching '*.css' under directory 'doc'
      warning: no previously-included files matching '*~' found anywhere in distribution
      warning: no previously-included files matching '*.pyc' found anywhere in distribution
      warning: no previously-included files matching '.svn' found anywhere in distribution
      no previously-included directories found matching 'doc/build'
      adding license file 'LICENSE'
      writing manifest file 'pygraphviz.egg-info/SOURCES.txt'
      copying pygraphviz/graphviz.i -> build/lib.macosx-14.1-arm64-cpython-312/pygraphviz
      copying pygraphviz/graphviz_wrap.c -> build/lib.macosx-14.1-arm64-cpython-312/pygraphviz
      running build_ext
      building 'pygraphviz._graphviz' extension
      creating build/temp.macosx-14.1-arm64-cpython-312
      creating build/temp.macosx-14.1-arm64-cpython-312/pygraphviz
      clang -fno-strict-overflow -Wsign-compare -Wunreachable-code -DNDEBUG -g -O3 -Wall -DSWIG_PYTHON_STRICT_BYTE_CHAR -I/tmp/venv/include -I/Users/jonathankaczynski/.pyenv/versions/3.12.0/include/python3.12 -c pygraphviz/graphviz_wrap.c -o build/temp.macosx-14.1-arm64-cpython-312/pygraphviz/graphviz_wrap.o
      pygraphviz/graphviz_wrap.c:3020:10: fatal error: 'graphviz/cgraph.h' file not found
      #include "graphviz/cgraph.h"
               ^~~~~~~~~~~~~~~~~~~
      1 error generated.
      error: command '/usr/bin/clang' failed with exit code 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for pygraphviz
Failed to build pygraphviz
ERROR: Could not build wheels for pygraphviz, which is required to install pyproject.toml-based projects

Attempt 2

Following the instructions on https://pygraphviz.github.io/documentation/stable/install.html, the installation still fails.

pip install --use-pep517 \
            --config-settings="--global-option=build_ext" \
            --config-settings="--build-option=-I$(brew --prefix graphviz)/include/" \
            --config-settings="--build-option=-L$(brew --prefix graphviz)/lib/" \
            pygraphviz

We don't see those build options in the clang command. Why?

clang -fno-strict-overflow -Wsign-compare -Wunreachable-code -DNDEBUG -g -O3 -Wall
        -DSWIG_PYTHON_STRICT_BYTE_CHAR
        -I/tmp/venv/include
        -I/Users/jonathankaczynski/.pyenv/versions/3.12.0/include/python3.12
        -c pygraphviz/graphviz_wrap.c
        -o build/temp.macosx-14.1-arm64-cpython-312/pygraphviz/graphviz_wrap.o
pygraphviz/graphviz_wrap.c:3020:10: fatal error: 'graphviz/cgraph.h' file not found

Details

(venv) jonathankaczynski@jkaczynski-mac ~ % pip install --use-pep517 \
        --config-settings="--global-option=build_ext" \
        --config-settings="--build-option=-I$(brew --prefix graphviz)/include/" \
        --config-settings="--build-option=-L$(brew --prefix graphviz)/lib/" \
        pygraphviz
Looking in indexes: https://pypi.org/simple
Collecting pygraphviz
  Using cached pygraphviz-1.12.tar.gz (104 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [11 lines of output]
      running build_ext
      building 'pygraphviz._graphviz' extension
      creating build
      creating build/temp.macosx-14.1-arm64-cpython-312
      creating build/temp.macosx-14.1-arm64-cpython-312/pygraphviz
      clang -fno-strict-overflow -Wsign-compare -Wunreachable-code -DNDEBUG -g -O3 -Wall -DSWIG_PYTHON_STRICT_BYTE_CHAR -I/tmp/venv/include -I/Users/jonathankaczynski/.pyenv/versions/3.12.0/include/python3.12 -c pygraphviz/graphviz_wrap.c -o build/temp.macosx-14.1-arm64-cpython-312/pygraphviz/graphviz_wrap.o
      pygraphviz/graphviz_wrap.c:3020:10: fatal error: 'graphviz/cgraph.h' file not found
      #include "graphviz/cgraph.h"
               ^~~~~~~~~~~~~~~~~~~
      1 error generated.
      error: command '/usr/bin/clang' failed with exit code 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Attempt 3

Only when we set the CFLAGS and LDFLAGS environment variables can we successfully install graphviz. This unblocks us, but it's not using the best practices, right?

Details

(venv) jonathankaczynski@jkaczynski-mac ~ % LDFLAGS="-L$(brew --prefix graphviz)/lib/" \
        CFLAGS="-I$(brew --prefix graphviz)/include/" \
        pip install pygraphviz
Looking in indexes: https://pypi.org/simple
Collecting pygraphviz
  Using cached pygraphviz-1.12.tar.gz (104 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: pygraphviz
  Building wheel for pygraphviz (pyproject.toml) ... done
  Created wheel for pygraphviz: filename=pygraphviz-1.12-cp312-cp312-macosx_14_0_arm64.whl size=92978 sha256=1e47b2e9c2fb8d2a5dfac7f103e34f52da537ca3fa94e9dd35028230995029ef
  Stored in directory: /Users/jonathankaczynski/Library/Caches/pip/wheels/3d/75/00/f9fa413fd48cfd0b0876bc81c475d6adfe7e3df369a15febf4
Successfully built pygraphviz
Installing collected packages: pygraphviz
Successfully installed pygraphviz-1.12

Output

No response

Code of Conduct

[X] I agree to follow the PSF Code of Conduct.

Feb 18 '24 18:02 jmkacz

For pip >= 23.0 I needed to switch to:

pip install \
    --config-settings="--global-option=build_ext" \
    --config-settings="--global-option=-I$(brew --prefix graphviz)/include/" \
    --config-settings="--global-option=-L$(brew --prefix graphviz)/lib/" \
    pygraphviz

rather than the documented:

pip install --use-pep517 \
            --config-settings="--global-option=build_ext" \
            --config-settings="--build-option=-I$(brew --prefix graphviz)/include/" \
            --config-settings="--build-option=-L$(brew --prefix graphviz)/lib/" \
            pygraphviz

Feb 27 '24 11:02 RUrlus