python-poetry
Allow for additional constraints on dependencies used in build environments
Issue Kind
Brand new capability
Description
When packages are installed via poetry install (or, I assume, poetry sync), in cases where a source distribution must be used, Poetry creates an isolated build environment, following requirements specified by the package. For various historical reasons, it's very common for the requirements to be, implicitly, "the latest version of Setuptools". Since Setuptools may have breaking changes, and may be updated after the release of the package to be installed, this carries the risk of causing the build to fail.
To avoid the problem, Poetry should provide some form of override to restrict the version of the build-time dependencies used to build the project's dependencies.
This could, for example, take the form of a new pyproject.toml section along the lines of
[tool.poetry.build-constraints]
some_dependency = { setuptools = "<78" }
Impact
The recent kerfuffle around https://github.com/pypa/setuptools/issues/4910 demonstrates clearly, IMO, that #4511 wasn't given sufficient consideration. Updating metadata for Python projects is difficult at the best of times, and there are still tons of legacy projects out there that don't even have a pyproject.toml file in which to specify a Setuptools version. (Trying to do so within setup.py is problematic, of course, because setuptools has to be imported before setup can be called.)
This isn't the first time there has been such a disruption; https://github.com/pypa/setuptools/issues/4519 was the same basic kind of problem appearing in Setuptools 72 (and, notably, both changes were relevant to Requests: https://github.com/psf/requests/issues/6775 https://github.com/psf/requests/pull/6920).
Going forward, Setuptools should be expected to implement (or at least try to implement, until community backlash forces a reversion) more removals of deprecated features - after all, that's what deprecation is for. Every new major version of Setuptools brings the risk of breaking large numbers of packages which don't upper-cap versions for their stated build dependencies. Even when those packages use a pyproject.toml, they might not have a [build-system] table, as long as Setuptools is the community-expected default (described as a lowercase "should" in PEP 517). This problem will therefore keep coming up as long as users don't have a clean way to override overly-permissive build requirement specifications.
This problem potentially impacts all build backends, not just Setuptools. While maintainers are often advised not to upper-cap Python dependencies, it's unclear whether this is good advice for build dependencies. Flit documentation, for example, advises using such a cap, but users are certainly not under any obligation to do so.
Workarounds
Pip can apparently handle this through an environment variable (e.g. https://github.com/pypa/setuptools/issues/4910#issuecomment-2748987218 ).
I implemented the proposal in #10388. Please try it out and give feedback if it works for you. (See https://python-poetry.org/docs/#installing-with-pipx for how to install Poetry from git.)
Just got burned by this again with https://github.com/pypa/setuptools/issues/5039
I think this would be extremely useful so we can work around these scenarios and pin the setuptools version to a known working one for specific packages.
~~I think there's still a quirk lingering in Poetry that's causing the above to be problematic (i need to open an issue for it), but having this feature would allow us to work around it.~~ The quirk ended up being a new version of cffi that uses metadata 2.4 format that an old install of poetry couldn't parse due to the age of pkginfo, so a dependency ended up not getting deployed into the isolated build environment and let to build failures for an old package that doesnt have a wheel. Oddly, when older versions of setuptools were used (pre 80.7.0) it found the missing dependency and created an egg so the build ended up working (note that this required the build environment to have pip installed). Newer versions of setuptools did not perform this stop gap. Regardless, i still think having this feature would be extremely useful.
