Improve interop with `importlib.metadata`: variation without `importlib_metadata`

[abravalheri]

trafficstars

This is basically the same as https://github.com/pypa/pyproject-hooks/pull/195 but without using importlib_metadata.

Feel free to close if #195 is preferred.

This might be worthy considering:

In general, I find that the following works relatively fine:

• using importlib_metadata.DistributionFinder, MetadataPathFinder to implement find_distrubitions, but then using importlib.metadata high-level APIs

And the following does not work:

• using importlib.metadata.DistributionFinder, MetadataPathFinder to implement find_distrubitions, but then using importlib_metadata high-level APIs

i.e. while this approach decreases the "spooky action at distance" of a transitive dependency installing importlib_metadata, it may also be more error prone if some dependency is indeed using importlib_metadata.

Closes #192

[takluyver]

Thanks! Does this somehow avoid the exception you found in setuptools tests in this comment?

[abravalheri]

For setuptools, it should be fine https://github.com/abravalheri/setuptools/pull/14 (tests should OK, docs failing but that is unrelated).

(I went ahead and already added fixed the importlib.metadata compatibility in setuptools: https://github.com/pypa/setuptools/pull/4339)

[takluyver]

Aha, so this is functionally similar to the original version of #195 (before involving importlib_metadata), but setuptools has changed so that it works?

I prefer this approach. I'll give it a day or so in case anyone else wants to weigh in, but if not then I'd like to merge this and cut another release.

[abravalheri]

(before involving importlib_metadata), but setuptools has changed so that it works?

It is very nuanced 😅

The reason for introducing importlib_metadata was mainly motivated by https://github.com/pypa/pyproject-hooks/pull/195#discussion_r1586395214.

Trying to import importlib_metadata alone does not fix setuptools (because of the vendoring drama).

If setuptools did not have to use vendoring, then yes using importlib_metadata would be enough and no change in setuptools would be necessary. That is related to the drawbacks and considerations discussed in https://github.com/pypa/pyproject-hooks/pull/199#issue-2275281951.

[ofek]

What are the next steps for this to be merged?

[abravalheri]

Hi @takluyver, @jaraco (and/or any other maintainers) is this approach (or the one in https://github.com/pypa/pyproject-hooks/pull/195 which includes importlib_metadata), a direction that the project is interested in pursuing? Or is the project interested in pursuing a different approach?

[jaraco]

I unfortunately have paged out the rich context, but I agree that it sounds as if this narrow and surgical fix may be a very solid improvement. Let's go with it for now. Thanks for the diligent investigation and proposal.

[nanonyme]

Would it be possible to make a release with this soon? The bug this aims to fix is basically update blocker for pyproject-hooks for any distro-like entities.

[icp1994]

And for any distro which has upgraded pyproject-hooks unaware of this cannot update affected packages - pex for example.

[jaraco]

@pradyunsg or @takluyver ^

[takluyver]

Sorry for the delay.

[takluyver]

v1.2 is tagged and on PyPI now - thanks for your patience and for the periodic reminders.

[henryiii]

1.2 has broken build's test suite when using backend-path on Python 3.8 and 3.9 (and check-sdist's setuptools test, though I just dropped the setuptools test there). See https://github.com/pypa/build/pull/820, https://github.com/python/importlib_metadata/issues/508, https://github.com/henryiii/check-sdist/pull/55. I've finally narrowed it down to this package's 1.2 release.

Backend path is ".", and setuptools is making test_no_prepare.egg-info, which is then getting picked up as a nameless distribution, which causes a confusing error when trying to get the name and falling back on None, then not handling the None case.

[takluyver]

:face_exhaling: Sorry.

I have no idea what's the right thing to do here now. #165 was meant to improve how an in-tree backend was loaded, but broke setuptools in some scenarios (#192). This PR was meant to fix that, but it seems that it's broken something in another scenario. :disappointed: Maybe we should have used importlib_metadata in some scenarios? But IIUC setuptools also has its own bundled copy, so that doesn't necessarily solve it.

I'm sorely tempted to declare that pyproject-hooks now requires Python 3.10 or above, and you should just pin whichever version works for you on older Python versions. I'm normally pretty keen on backwards compatibility, so I'd welcome a better solution, but I don't have the time or energy to wade through importlib stuff to figure this out. And I don't know how we can trust that the next solution won't just lead to another of these bugs popping up somewhere.

[pradyunsg]

Let's have the discussion on that in #206.