Release 1.1: Regression breaks setuptools builds

[jaraco]

In https://github.com/pypa/setuptools/issues/4333, setuptools reports some emergent failures implicating the recent pyproject-hooks 1.1 release.

[jaraco]

I suspect this change is relevant.

• More careful handling of the backend-path key from pyproject.toml. Previous versions would load the backend and then check that it was loaded from the specified path; the new version only loads it from the specified path. The BackendInvalid exception is now a synonym for :exc:BackendUnavailable, and code should move to using the latter name.

[pradyunsg]

https://github.com/pypa/pyproject-hooks/pull/165 is the relevant PR -- /cc @abravalheri for awareness.

It's unclear to me what exactly is happening here from the setuptools failure, so not quite sure what the fix should/would be?

[jaraco]

My initial thought is that setuptools' builds were previously relying on metadata from an existing installation... and this change keeps that metadata hidden so that all that's around is the bootstrap metadata, which isn't sufficient to build setuptools (maybe?).

[jaraco]

Or, maybe the changes to avoid altering sys.path cause the metadata to be missing altogether.

[pradyunsg]

The sys.path with 1.0.0:

['/Users/pradyunsg/Developer/github/setuptools', '/Users/pradyunsg/Developer/github/setuptools/.venv/lib/python3.12/site-packages/pyproject_hooks/_in_process', '/Users/pradyunsg/.asdf/installs/python/3.12.0/lib/python312.zip', '/Users/pradyunsg/.asdf/installs/python/3.12.0/lib/python3.12', '/Users/pradyunsg/.asdf/installs/python/3.12.0/lib/python3.12/lib-dynload', '/Users/pradyunsg/Developer/github/setuptools/.venv/lib/python3.12/site-packages']

The sys.path with 1.1.0:

['/Users/pradyunsg/Developer/github/setuptools/.venv/lib/python3.12/site-packages/pyproject_hooks/_in_process', '/Users/pradyunsg/.asdf/installs/python/3.12.0/lib/python312.zip', '/Users/pradyunsg/.asdf/installs/python/3.12.0/lib/python3.12', '/Users/pradyunsg/.asdf/installs/python/3.12.0/lib/python3.12/lib-dynload', '/Users/pradyunsg/Developer/github/setuptools/.venv/lib/python3.12/site-packages']

It's... suboptimal... that _in_process is on sys.path with this setup.

[pradyunsg]

Yup, it's the changeover to using sys.meta_path rather than sys.path for loading the backend that's affected things here.

[jaraco]

I suspect adding a find_distributions (i.e. making it an importlib.metadata.DistributionFinder) will allow it to find the metadata for a backend and will likely address the concern.

[mdagost]

I also suspect this is the root cause of this issue: https://github.com/python-poetry/poetry/issues/9351

[pradyunsg]

@mdagost No. See #191.

[mdagost]

Yes. Simply downgrading pyproject-hooks from 1.1.0 to 1.0.0 with no other changes fixes poetry install for all our projects.

[mdagost]

Sorry--update. I believe the root cause is that 1.0.0 was locked in the poetry.lock file, but then poetry itself pulled in 1.1.0 when it was pip installed. So there was a conflict. Either downgrading to 1.0.0 after pip installing, or upgrading to 1.1.0 in the lock file fixes things for me.

[pradyunsg]

@mdagost That sounds unrelated to this project, and specific to how Poetry is handling its environment and lockfile.

Please, let's keep any further Poetry-specific discussion in the Poetry issue tracker or the other issue in this repo.

[mdagost]

Correct. The root root cause was managing poetry with the lockfile, which is no bueno.

[jacobfriedman]

CPython should stop shipping with prebuilt pip and invoking an install of prebuilt setuptools. Even Fedora has a problem with this.

[nanonyme]

@jacobfriedman CPython shipping pip has absolutely nothing to do with this problem, this same issue happens with pyproject stack with no pip and only build+installer.