distutils icon indicating copy to clipboard operation
distutils copied to clipboard

Published 20 hours ago • verified publisher icon pypa

Cygwin currently does not support --enable-new-dtags

Open Joshua-IRT opened this issue 2 years ago • 2 comments

setuptools version

63.2.0

Python version

Python 3.9.10

OS

Cygwin on Windows 10

Additional environment information

No response

Description

I have installed Python 3.9, pip, setuptools (59.5.0) and proj from the Cygwin package manager. Attempting to install pyproj using pip results in pip collecting the latest setuptools and trying to build pyproj, but failing with an unknown flag:

/usr/lib/gcc/x86_64-pc-cygwin/11/../../../../x86_64-pc-cygwin/bin/ld: unrecognized option '--enable-new-dtags'

This has been patched in the version of distutils included within Cygwin's Python 3.9 build but not the setuptools available in the Cygwin installer, nor the upstream setuptools from pypi.

Expected behavior

I expect pyproj to build and install successfully.

How to Reproduce

  1. Install Cygwin, Python 3.9, pip, setuptools and proj using the Cygwin package installer.
  2. Try to install pyproj using pip.

Output

$ pip3 install pyproj -v
Using pip 21.3.1 from /usr/lib/python3.9/site-packages/pip (python 3.9)
Collecting pyproj
  Using cached pyproj-3.3.1.tar.gz (219 kB)
  Running command /usr/bin/python3.9.exe /tmp/pip-standalone-pip-52gtcyvv/__env_pip__.zip/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-aszmv_fv/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- setuptools wheel 'cython>=0.28.4'
  Collecting setuptools
    Using cached setuptools-63.2.0-py3-none-any.whl (1.2 MB)
  Collecting wheel
    Using cached wheel-0.37.1-py2.py3-none-any.whl (35 kB)
  Collecting cython>=0.28.4
    Using cached Cython-0.29.30-py2.py3-none-any.whl (985 kB)
  Installing collected packages: wheel, setuptools, cython
  ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
  ipython 7.30.1 requires jedi>=0.16, which is not installed.
  ipython 7.30.1 requires matplotlib-inline, which is not installed.
  Successfully installed cython-0.29.30 setuptools-63.2.0 wheel-0.37.1
  WARNING: You are using pip version 21.3.1; however, version 22.2 is available.
  You should consider upgrading via the '/usr/bin/python3.9.exe -m pip install --upgrade pip' command.
  Installing build dependencies ... done
  Running command /usr/bin/python3.9.exe /usr/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py get_requires_for_build_wheel /tmp/tmpucwx7m3f
  /tmp/pip-build-env-aszmv_fv/overlay/lib/python3.9/site-packages/setuptools/config/setupcfg.py:463: SetuptoolsDeprecationWarning: The license_file parameter is deprecated, use license_files instead.
    warnings.warn(msg, warning_class)
  running egg_info
  writing pyproj.egg-info/PKG-INFO
  writing dependency_links to pyproj.egg-info/dependency_links.txt
  writing entry points to pyproj.egg-info/entry_points.txt
  writing requirements to pyproj.egg-info/requires.txt
  writing top-level names to pyproj.egg-info/top_level.txt
  reading manifest file 'pyproj.egg-info/SOURCES.txt'
  reading manifest template 'MANIFEST.in'
  warning: no files found matching 'pyproj/*.pyd'
  no previously-included directories found matching 'docs/_build'
  no previously-included directories found matching 'pyproj/proj_dir'
  adding license file 'LICENSE'
  writing manifest file 'pyproj.egg-info/SOURCES.txt'
  Getting requirements to build wheel ... done
  Running command /usr/bin/python3.9.exe /usr/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py prepare_metadata_for_build_wheel /tmp/tmp9570589p
  /tmp/pip-build-env-aszmv_fv/overlay/lib/python3.9/site-packages/setuptools/config/setupcfg.py:463: SetuptoolsDeprecationWarning: The license_file parameter is deprecated, use license_files instead.
    warnings.warn(msg, warning_class)
  running dist_info
  creating /tmp/pip-modern-metadata-8e405tu3/pyproj.egg-info
  writing /tmp/pip-modern-metadata-8e405tu3/pyproj.egg-info/PKG-INFO
  writing dependency_links to /tmp/pip-modern-metadata-8e405tu3/pyproj.egg-info/dependency_links.txt
  writing entry points to /tmp/pip-modern-metadata-8e405tu3/pyproj.egg-info/entry_points.txt
  writing requirements to /tmp/pip-modern-metadata-8e405tu3/pyproj.egg-info/requires.txt
  writing top-level names to /tmp/pip-modern-metadata-8e405tu3/pyproj.egg-info/top_level.txt
  writing manifest file '/tmp/pip-modern-metadata-8e405tu3/pyproj.egg-info/SOURCES.txt'
  reading manifest file '/tmp/pip-modern-metadata-8e405tu3/pyproj.egg-info/SOURCES.txt'
  reading manifest template 'MANIFEST.in'
  warning: no files found matching 'pyproj/*.pyd'
  no previously-included directories found matching 'docs/_build'
  no previously-included directories found matching 'pyproj/proj_dir'
  adding license file 'LICENSE'
  writing manifest file '/tmp/pip-modern-metadata-8e405tu3/pyproj.egg-info/SOURCES.txt'
  creating '/tmp/pip-modern-metadata-8e405tu3/pyproj-3.3.1.dist-info'
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: certifi in /usr/local/lib/python3.9/site-packages/certifi-2022.6.15-py3.9.egg (from pyproj) (2022.6.15)
Building wheels for collected packages: pyproj
  Running command /usr/bin/python3.9.exe /usr/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py build_wheel /tmp/tmpwdas1qdy
  /tmp/pip-build-env-aszmv_fv/overlay/lib/python3.9/site-packages/setuptools/config/setupcfg.py:463: SetuptoolsDeprecationWarning: The license_file parameter is deprecated, use license_files instead.
    warnings.warn(msg, warning_class)
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.cygwin-3.3.5-x86_64-cpython-39
  creating build/lib.cygwin-3.3.5-x86_64-cpython-39/pyproj
  copying pyproj/aoi.py -> build/lib.cygwin-3.3.5-x86_64-cpython-39/pyproj
  copying pyproj/datadir.py -> build/lib.cygwin-3.3.5-x86_64-cpython-39/pyproj
  copying pyproj/enums.py -> build/lib.cygwin-3.3.5-x86_64-cpython-39/pyproj
  copying pyproj/exceptions.py -> build/lib.cygwin-3.3.5-x86_64-cpython-39/pyproj
  copying pyproj/geod.py -> build/lib.cygwin-3.3.5-x86_64-cpython-39/pyproj
  copying pyproj/network.py -> build/lib.cygwin-3.3.5-x86_64-cpython-39/pyproj
  copying pyproj/proj.py -> build/lib.cygwin-3.3.5-x86_64-cpython-39/pyproj
  copying pyproj/sync.py -> build/lib.cygwin-3.3.5-x86_64-cpython-39/pyproj
  copying pyproj/transformer.py -> build/lib.cygwin-3.3.5-x86_64-cpython-39/pyproj
  copying pyproj/utils.py -> build/lib.cygwin-3.3.5-x86_64-cpython-39/pyproj
  copying pyproj/_show_versions.py -> build/lib.cygwin-3.3.5-x86_64-cpython-39/pyproj
  copying pyproj/__init__.py -> build/lib.cygwin-3.3.5-x86_64-cpython-39/pyproj
  copying pyproj/__main__.py -> build/lib.cygwin-3.3.5-x86_64-cpython-39/pyproj
  creating build/lib.cygwin-3.3.5-x86_64-cpython-39/pyproj/crs
  copying pyproj/crs/coordinate_operation.py -> build/lib.cygwin-3.3.5-x86_64-cpython-39/pyproj/crs
  copying pyproj/crs/coordinate_system.py -> build/lib.cygwin-3.3.5-x86_64-cpython-39/pyproj/crs
  copying pyproj/crs/crs.py -> build/lib.cygwin-3.3.5-x86_64-cpython-39/pyproj/crs
  copying pyproj/crs/datum.py -> build/lib.cygwin-3.3.5-x86_64-cpython-39/pyproj/crs
  copying pyproj/crs/enums.py -> build/lib.cygwin-3.3.5-x86_64-cpython-39/pyproj/crs
  copying pyproj/crs/_cf1x8.py -> build/lib.cygwin-3.3.5-x86_64-cpython-39/pyproj/crs
  copying pyproj/crs/__init__.py -> build/lib.cygwin-3.3.5-x86_64-cpython-39/pyproj/crs
  copying pyproj/database.pyi -> build/lib.cygwin-3.3.5-x86_64-cpython-39/pyproj
  copying pyproj/list.pyi -> build/lib.cygwin-3.3.5-x86_64-cpython-39/pyproj
  copying pyproj/_compat.pyi -> build/lib.cygwin-3.3.5-x86_64-cpython-39/pyproj
  copying pyproj/_crs.pyi -> build/lib.cygwin-3.3.5-x86_64-cpython-39/pyproj
  copying pyproj/_datadir.pyi -> build/lib.cygwin-3.3.5-x86_64-cpython-39/pyproj
  copying pyproj/_geod.pyi -> build/lib.cygwin-3.3.5-x86_64-cpython-39/pyproj
  copying pyproj/_network.pyi -> build/lib.cygwin-3.3.5-x86_64-cpython-39/pyproj
  copying pyproj/_sync.pyi -> build/lib.cygwin-3.3.5-x86_64-cpython-39/pyproj
  copying pyproj/_transformer.pyi -> build/lib.cygwin-3.3.5-x86_64-cpython-39/pyproj
  copying pyproj/py.typed -> build/lib.cygwin-3.3.5-x86_64-cpython-39/pyproj
  running build_ext
  building 'pyproj._geod' extension
  creating build/temp.cygwin-3.3.5-x86_64-cpython-39
  creating build/temp.cygwin-3.3.5-x86_64-cpython-39/pyproj
  gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -I/usr/include -I/usr/include/python3.9 -c pyproj/_geod.c -o build/temp.cygwin-3.3.5-x86_64-cpython-39/pyproj/_geod.o
  gcc -shared -Wl,--enable-auto-image-base build/temp.cygwin-3.3.5-x86_64-cpython-39/pyproj/_geod.o -L/usr/lib -L/usr/lib/python3.9/config -L/usr/lib -Wl,--enable-new-dtags,-R/usr/lib -lproj -lpython3.9 -o build/lib.cygwin-3.3.5-x86_64-cpython-39/pyproj/_geod.cpython-39-x86_64-cygwin.dll
  /usr/lib/gcc/x86_64-pc-cygwin/11/../../../../x86_64-pc-cygwin/bin/ld: unrecognized option '--enable-new-dtags'
  /usr/lib/gcc/x86_64-pc-cygwin/11/../../../../x86_64-pc-cygwin/bin/ld: use the --help option for usage information
  collect2: error: ld returned 1 exit status
  error: command '/usr/bin/gcc' failed with exit code 1
  Building wheel for pyproj (pyproject.toml) ... error
  ERROR: Failed building wheel for pyproj
Failed to build pyproj
ERROR: Could not build wheels for pyproj, which is required to install pyproject.toml-based projects

Joshua-IRT avatar Jul 26 '22 08:07 Joshua-IRT

Setuptools has adopted distutils from the stdlib. Therefore, it's necessary for any cygwin-specific features to gain native support in distutils. It's no longer viable for cygwin to monkeypatch distutils. I'll transfer this issue to pypa/distutils (where contributions will later be merged back into Setuptools). Please find someone who can analyze and describe what --enable-new-dtags is meant to do and propose a way for distutils to implement it.

jaraco avatar Aug 14 '22 23:08 jaraco

I've usually seen --enable-new-dtags right before --rpath, which isn't a thing on Cygwin; probably some extension tries to set runtime_library_dir, which is a Linux-specific thing (maybe all ELF platforms, so perhaps also BSD).

It looks like pyproj already has code to not specify runtime_library_dir on Windows; extending that to also check for Cygwin should fix the issue. Updating setuptools to the most recent version (63.1) should produce a setuptools warning instead of a compiler error; updating to git trunk should provide a more descriptive message in the warning. However, pip installs its own setuptools==63.2.0 because pyproj has a pyproject.toml file; not sure why this isn't triggering the warning and avoiding the error.

EDIT: The warning isn't triggering on Cygwin because the warning is in CygwinCCompiler, while distutils uses UnixCCompiler on Cygwin. There's discussion in an issue (#185) and PR (#209) over whether we should delete CygwinCCompiler as unused or start using CygwinCCompiler, maybe deleting most of its contents since it inherits from UnixCCompiler and that's mostly been working fine.

DWesl avatar Aug 16 '22 13:08 DWesl