coveragepy icon indicating copy to clipboard operation
coveragepy copied to clipboard

Use PEP 669 Low Impact Monitoring to reduce overhead

Open intgr opened this issue 1 year ago • 3 comments

Is your feature request related to a problem? Please describe. coveragepy adds significant overhead to test execution in CI.

And this got much worse with Python 3.12 due to CPython performance regression (#1665, https://github.com/python/cpython/issues/107674, maybe will be fixed by https://github.com/python/cpython/pull/107841)

Describe the solution you'd like PEP 669 – Low Impact Monitoring for CPython (sys.monitoring) should be explored to see if it claws back lost performance or even results in performance gains. Though AFAIK it is not proven yet whether it will help.

Additional context As I understand, this is already under development. But I wanted to open an issue anyway to track progress. In https://github.com/nedbat/coveragepy/issues/1665#issuecomment-1778042967 (Oct 25, 2023) nedbat states:

I have started on code to use the new PEP 669 monitoring, but I haven't made much progress on it. I'd welcome help.

intgr avatar Feb 09 '24 10:02 intgr

sys.monitoring support is in coverage.py 7.4.0, though it only makes line coverage faster, not branch coverage. I'm still working on the support that will be faster for branch coverage.

nedbat avatar Feb 10 '24 11:02 nedbat

sys.monitoring support is in coverage.py 7.4.0

Awesome, thanks! I'll test COVERAGE_CORE=sysmon on monday. I was completely oblivious to that. I hadn't read the changelog and it wasn't mentioned in any of the threads that came up from my Google search.

intgr avatar Feb 10 '24 17:02 intgr

Indeed COVERAGE_CORE=sysmon reduced test suite runtime by ~35%, compared to Python 3.12. That's ~20% faster than Python 3.11 even.

intgr avatar Feb 12 '24 09:02 intgr