Allow setting build env constraints

[webknjaz]

So normally folks put direct unrestricted dependencies into their requires under [build-system] in pyproject.toml. This means that building the same dist may become unpredictable over time. One way to solve this is to somehow generate a dependency tree with all transitive deps pinned and stick it there.

But this approach seems weird. Do you think it'd be reasonable to have something like python -m build ... --constraints=build-env-deps.txt pointing to constraints for use in pip install command?

[FFY00]

This is out of scope IMO. This package aims to be simple, as it is crucial for bootstraping Python environments, so I'd like to avoid bringing much complexity into it. It is a hard line to draw, and this use-case does not seem strong enough. We already provide all the CLI, or API, required for making this work, so not having the glue directly available in the project should not be an issue.

You can acheive the same with the following commands.

pip install -r contraints.txt
python -m build -n

[webknjaz]

The idea here is to avoid using constraints as requirements, because they are not that, though.

[gaborbernat]

I feel like this should be discussed at the PEP level, not the tool level. Currently, PEP-518/517 does not handle your request.

[FFY00]

The idea here is to avoid using constraints as requirements, because they are not that, though.

I think I don't really understand this comment. Could you clarify?

[layday]

Are you able to pass constraints in an env var? Does that work with pip?

[uranusjr]

Yes I think setting PIP_INSTALL_CONSTRAINTS should work (whether that’s a proper feature is up to debate).

[webknjaz]

Yes I think setting PIP_INSTALL_CONSTRAINTS should work (whether that’s a proper feature is up to debate).

Woah, I haven't thought about this. I guess having such a workaround would be useful. Although, for pip install scenario (unrelated to build) it'd probably cause multiple transitive deps in the tree to fail the build...

I think I don't really understand this comment. Could you clarify?

I meant that I'd like to pass --constraints to pip like @layday and @uranusjr while having no pins in [build-system.requires].

[webknjaz]

So while reflecting on this, I now have a slightly different feature idea for build.

Would it be possible to have a command for extracting the list of the build deps from toml? The idea is that if there's something like build --export-sdist-build-deps-to=build-requirements.txt, this would facilitate the manual build env provisioning process w/o having to have an additional to build config parsing implementation.

[layday]

The issue with that is that it might be incomplete if the backend defines extra dependencies in get_requires_for_build_sdist and you can’t call the hook without having previously installed the pyproject.toml dependencies.

Sent with ProtonMail Secure Email.

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Tuesday, 11 May 2021 01:32, Sviatoslav Sydorenko @.***> wrote:

So while reflecting on this, I now have a slightly different feature idea for build.

Would it be possible to have a command for extracting the list of the build deps from toml? The idea is that if there's something like build --export-sdist-build-deps-to=build-requirements.txt, this would facilitate the manual build env provisioning process w/o having to have an additional to build config parsing implementation.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[webknjaz]

Yes I think setting PIP_INSTALL_CONSTRAINTS should work (whether that’s a proper feature is up to debate).

I've recently verified that setting PIP_CONSTRAINT works: https://github.com/cherrypy/cheroot/commit/5c1ac088c595ec91b9bda2dc818e491a23132a2c#diff-ef2cef9f88b4fe09ca3082140e67f5ad34fb65fb6e228f119d3812261ae51449R51.

[nitzmahone]

So long as the envvar passes all the way through to the actual build environment when the build deps are installed, that's probably good enough for now, but yeah, agreed that this is a common problem that we've been nailed on numerous times, and just manually syncing the requirements is a lot more problematic than constraining known problem cases. It seems like this is already a problem that every backend needs to solve, so long-term, having a common way to do it would be a Good Thing.

[layday]

I don't see a future where build allows you to constrain your build deps the way it's set up currently - that sounds like the job of a locker/env manager, which build is not. If build had a more pluggable architecture and you could write 'drivers' for pip/Poetry/pdm/pip-lock/who knows what, then it could make sense. But I can't imagine who's gonna make that happen or if build is the place where it should happen.

[webknjaz]

I think that the locker thing is separate, yes. I use pip-tools to produce the constraints file. But what is the problem with making pip consume it? It's already being called under the hood anyway.

[joerick]

I'd like to add a +1 to this feature request.

build makes a virtualenv and installs the packages listed in build-system.requires before invoking the build-backend. I think it should be possible to specify a pin/constraint for these packages at runtime.

There are a couple of use-cases.

• Build repeatability users like @webknjaz above want more determinism in their builds, for consistency, predictability, easier use in CI, easier debugging, reduced supply-chain attack surface area.

  For this use-case, we might argue that you can get around this by pinning the versions of tools in build-system.requires. But this is not ideal, because it guarantees that the package cannot move with the times, for a new version of Python for example. And, it's not possible to pin to a specific SHA-256 digest this way (something that was noted in this recent packaging complain-o-post).

• Embedding into other tools (selfishly!) cibuildwheel has been using the PIP_CONSTRAINTS workaround for a few years now to ensure that if a user pins a version of cibuildwheel, then the versions of build tools like pip and setuptools remain static for repeatability. This is accomplished via PIP_CONSTRAINTS too, but it means that users can't do their own pinning too.

The solution above, using PIP_CONSTRAINTS is really working around the absence of this feature by reaching into the internals of pypa/build and configuring its pip invocation. This happens to work, but it risks hitting other invocations of pip, and is unsupported. There is no other way to make the build repeatable, without disabling build isolation.

I'd like to propose a command line option, like-

build ...
    [--dependency-constraints FILE]
    ....

Options:
...
  --dependency-constraints FILE   Use a pip constraints file to control versions
                                  of packages in build-system.requires. Can be
                                  used multiple times.
    

Which could be routed through to the pip install line at https://github.com/pypa/build/blob/main/src/build/env.py#L133 .

I understand that the maintainers here want to keep the tool small, but given that build is doing the job of installing the build-env dependencies, it seems natural to give users control over this small part, before the build-backend takes over.