v3.4 sdist contains empty setup.py

[jayvdb]

Hi, https://files.pythonhosted.org/packages/8b/e1/43beb3d38dba6cb420cefa297822eac205a277ab43e5ba5d5c46faf96438/idna-3.4.tar.gz contains a setup.py which is:

from setuptools import setup
setup()

There is no setup.cfg , so as a result python setup.py build works, but it installs using version '0.0.0' which then causes problems with pkg_resources. e.g. pkg_resources.ContextualVersionConflict: (idna 0.0.0 (/usr/lib/python3.10/site-packages), Requirement.parse('idna>=2.5'), {'hyperlink'}) /usr/lib/python3.10/site-packages/pkg_resources/init.py:800: ContextualVersionConflict

[kjd]

Thanks. It's been suggested by another contributor to completely removed setup.py and instead rely on pyproject.toml. Do you need setup.py to be retained for your needs or would this solve the problem?

[jayvdb]

From a distro packaging perspective, having a setup.py/.cfg for idna would be slightly beneficial, as it allows more parallelization in the process of rebuilding the entire python package collection, because idna can then be built much earlier as it only depends on setuptools. However at least you've chosen flit which has only a few deps compared to poetry, etc.

Another aspect to consider is that if building idna depend on flit, the number of distros that can include idna is limited by https://repology.org/project/python:flit/versions . As a practical example, openSUSE Leap 14.3 (the latest) has a very old version of flit which doesnt work very well. If there is only a dependency on setuptools, the ease of packaging is easier due to https://repology.org/project/python:setuptools/versions .

OTOH, not having a setup.py/cfg is easier for you. That is also important. A perfect solution would be using setuptools 's pyproject.toml support, but it is more limited than flit/etc.

[hgoldfish]

without setup.py or setup.cfg this command will go wrong under python 3.7 debian 10

pip install --no-binary :all: idna==3.4

[Odilhao]

I'm facing the same problem on CentosStream8

running install
running install_egg_info
running egg_info
creating UNKNOWN.egg-info
writing UNKNOWN.egg-info/PKG-INFO
writing dependency_links to UNKNOWN.egg-info/dependency_links.txt
writing top-level names to UNKNOWN.egg-info/top_level.txt
writing manifest file 'UNKNOWN.egg-info/SOURCES.txt'
reading manifest file 'UNKNOWN.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no previously-included files matching '*.pyc' found under directory 'tools'
warning: no previously-included files matching '*.pyc' found under directory 'tests'
writing manifest file 'UNKNOWN.egg-info/SOURCES.txt'
Copying UNKNOWN.egg-info to /builddir/build/BUILDROOT/python-idna-3.4-1.el8.noarch/usr/lib/python3.9/site-packages/UNKNOWN-3.4-py3.9.egg-info
running install_scripts

The UNKNOWN-3.4-py3.9.egg-info only appears after forcing use_scm_version=True inside the setup.py with:

sed -i 's/setup()/setup(use_scm_version=True)/' setup.py

Would be nice to have one minimal setup.cfg.

[kjd]

@jayvdb @hgoldfish @sethmlarson I've created a branch with a setup.cfg file. Does this file address the reported problems? Does it have any side effects? Are there any alternative approaches to consider?

[Odilhao]

@kjd thanks for providing the setup.cfg, I just tested on CentosStream 8 and it worked after adding:

[options]
python_requires = >=3.5
+packages=find:

Without the packages=find: it just installed the following files:

/usr/lib/python3.9/site-packages/idna-3.4-py3.9.egg-info
/usr/lib/python3.9/site-packages/idna-3.4-py3.9.egg-info/PKG-INFO
/usr/lib/python3.9/site-packages/idna-3.4-py3.9.egg-info/SOURCES.txt
/usr/lib/python3.9/site-packages/idna-3.4-py3.9.egg-info/dependency_links.txt
/usr/lib/python3.9/site-packages/idna-3.4-py3.9.egg-info/top_level.txt
/usr/share/doc/python39-idna
/usr/share/doc/python39-idna/README.rst
/usr/share/licenses/python39-idna
/usr/share/licenses/python39-idna/LICENSE.md

The only side effect that I noticed is that now the tests dir is not being excluded by default during the build.

[sethmlarson]

PEP 517/pyproject.toml (finalized in 2017) is the future of Python packaging, I don't think Python packages should be duplicating metadata in multiple places knowing that it will become out of date due to not being on the critical path or causing confusion for future contributors. The correct fix here would be for downstream repackaging tools to adopt PEP 517 so packages can move to the correct standard.

idna isn't the only package moving to PEP 517, so if these platforms need to continue consuming Python packages they will need to migrate eventually.

[kjd]

This project has taken a very conservative approach to date of continuing to support older versions of Python given how many other applications depend on it, often indirectly without explicitly knowing they do. It would seem consistent with this approach to ease problems for significant downstream repackagers if its not too burdensome.

With that said, this has presumably been a problem for about a year, and this package reportedly gets downloaded about 5 million times a day from PyPI. Given the relative lack of bug reports it seems the impact is pretty low of not having the setup.cfg in place? I agree it is not elegant and something to be careful about if we duplicate the package information in two places. To Seth's point maybe this is a good nudge if this is the way packaging needs to go anyway.

The next Unicode release is scheduled for 12 September, and I'd like to cut the next release shortly thereafter, so I'd like to get consensus on the best approach here by then.

[kjd]

I've temporarily re-added setup.cfg for this next release because I haven't heard a significant downside, but I think the maintainers of other projects that rely on it are on notice and it seems inevitable it will be removed again. It would be good if there are updates on whether these downstream dependencies still have an issue with the absence of setup.cfg, and if they have bug reports open to fix.

[kjd]

v3.5 has been published which reintroduces setup.py for now. Issue #155 has been opened to track removing it at a future date. Please keep us posted on any significant downstream dependencies that still require it.