build icon indicating copy to clipboard operation
build copied to clipboard

Consider turning off coverage for PyPy tests

Open layday opened this issue 4 years ago • 6 comments

Coverage (i.e. tracing) is supposed to slow down execution considerably on PyPy. We could try turning it off for the PyPy tests to see how much of a difference it makes.

layday avatar Oct 15 '21 18:10 layday

The main issue seems to be Windows runs with coverage, the others are fine IMO, they can range from more or less the same duration to a bit slower, but Windows runs are the vast majority of the time the bottleneck. So, I think we can just turn off coverage on Windows. OTOH, CI is free, and we have auto-merge so :shrug:

FFY00 avatar Oct 15 '21 19:10 FFY00

Well, for one, slow-running tasks can frustrate contributors. Repeat pushes will place runs in a FIFO queue and a contributor will have to wait for hours for CI to finish. I often find myself cancelling runs from the Actions tab. And even though CI might be gratis, when runs are this slow, you begin to wonder how wasteful they might really be.

but Windows runs are the vast majority of the time the bottleneck

We can also think about only running coverage on the latest stable (C)Python version.

layday avatar Oct 15 '21 20:10 layday

We can add concurrently groups and auto cancel if you'd like.

henryiii avatar Oct 15 '21 20:10 henryiii

We can add concurrently groups and auto cancel if you'd like.

I was meaning to do this. I think it would be a good first step, and we can then assess if/where it is worth disabling coverage.

FFY00 avatar Oct 15 '21 20:10 FFY00

Wouldn't it mean that we'd not get code coverage for the pypy branches? :thinking:

gaborbernat avatar Oct 15 '21 21:10 gaborbernat

I'm doing that in random other places currently, might as well add it here too. (Literally still in my clipboard) :)

henryiii avatar Oct 15 '21 21:10 henryiii