pip icon indicating copy to clipboard operation
pip copied to clipboard

Add packse (Python packaging scenarios) to Pip's test suite

Open notatallshaw opened this issue 1 year ago • 2 comments

What's the problem this feature will solve?

It seems astral has spun out a specific repository for Python packaging scenarios: https://github.com/zanieb/packse

Describe the solution you'd like

Perhaps all these scenarios are already covered by Pip, but it seems like it would do no harm to run these during testing?

Alternative Solutions

Analyse and convert these manually to Pip's test suite

Additional context

I only just found out about this repo, so I don't know what the lift is to try and use it for Pip testing.

But I would be happy to investigate and see if I can make a PR some time unless there is major objection from Pip devs. Would be at least a couple of months until I could spend time on this, so I'm not volunteering myself to lead this immediately.

Code of Conduct

notatallshaw avatar Feb 16 '24 18:02 notatallshaw

I'd be happy to have them covered somehow, but IMO we need to be careful not to significantly increase the runtime of the test suite any further - our tests are already sufficiently slow that it's a problem.

pfmoore avatar Feb 16 '24 19:02 pfmoore

Happy to help out with this in whatever way possible, just let me know.

zanieb avatar Feb 17 '24 16:02 zanieb

Instead of doing this as a part of the default CI, maybe we can run this periodically instead (on a less frequent schedule) and create an issue if it fails?

uranusjr avatar Mar 12 '24 22:03 uranusjr

Let's see how long it will take first, if it's less than a minute it seems like it would be worth catching on each PR.

The CI tooling already missed that removing setup.py broke many tests due to not running.

notatallshaw avatar Mar 12 '24 23:03 notatallshaw

Okay, I gave this a try and I have this experimentally working with pip.

Using a very naive check I have 26 failing tests and 66 passing tests, but I have not done any debugging yet to see if I'm doing something wrong, so I may have a draft PR ready to go this weekend.

The total time on my PC takes 67 seconds to run and requires Python 3.12+. I am considering making it it's own seperate runner, as I don't think the tests need to run against every platform?

notatallshaw avatar Mar 16 '24 01:03 notatallshaw

Okay, I've created a draft PR: https://github.com/pypa/pip/pull/12580

There's still work to do on some scenarios/tests, but I think the general approach is there.

I would appreciate feedback from both pip and packse maintainers.

notatallshaw avatar Mar 17 '24 15:03 notatallshaw