Clarify future for code coverage from subprocesses

[paarth-a]

Hi folks,

With the release of v7.0.0, support for out-of-the-box code coverage of subprocesses was removed in favour of an opt-in config option.

What does the future of code coverage creation from subprocesses look like? Are there plans to deprecate the config option (patch = ["subprocess"] or similar) in the longer term?

Thanks for maintaining this plugin and for any clarity you can provide on the long-term plans for subprocess coverage.

[nedbat]

patch = subprocess is a coverage.py configuration option that was just added. Maybe you meant to ask about something else being deprecated?

[paarth-a]

patch = subprocess is a coverage.py configuration option that was just added. Maybe you meant to ask about something else being deprecated?

Regardless of the underlying implementation changes, shifting the default to an opt-in can sometimes suggest a more cautious stance or possible reevaluation of the feature’s long-term role.

If the intention is simply to move this functionality out of pytest-cov and into coverage.py, would the pytest-cov team consider enabling the subprocess patch by default in a future release?

[ionelmc]

I removed the builtin support for suprocesses in pytest-cov because it was creating interaction issues with the various patches that coverage has. If you wanna dive in deeper: https://github.com/pytest-dev/pytest-cov/issues/708

The reason I didn't enable that by default is because I didn't have a nice way to opt-out from it and I wasn't convinced it's the right choice. Need to discuss if it's worth having yet another cli option to opt-out (i believe is necessary to implement what you want) vs asking users to just opt-in via coverage config (the current situation).

Also, in my projects that I want to get subprocess coverage I have a coverage config. Actually I have a config file in all of them. Not really a big deal to add "patch=subprocess" when I need it. The way I thought about it is I don't want to enable a patch that I don't need - it's just extra complexity.

Do you use pytest-cov without any coverage config file? If you have one, is it a too big ask to just have "patch=subprocess" in it?

[nedbat]

@ionelmc Thanks for working this through with us.

The reason I didn't enable that by default is because I didn't have a nice way to opt-out from it

I thought the behavior in pytest-cov 6.x was that it always applied its own subprocess patch. Can you say more about why we need a way to opt-out of subprocess support now? I'm probably missing some subtlety.

[ionelmc]

Well if I enable the subprocess patch by default then I think someone would want to want opt-out, or find yet another interaction issue, just like you found out in #708 - the "less complexity" line of thought.

[ionelmc]

To explain more, to fix #708 I could have added an opt-out from the builtin subprocess stuff in pytest-cov but when I tried patch=subprocess I found out that it worked so well - I didn't have any good reason to still maintain what was really 15 years old code.

[paarth-a]

Also, in my projects that I want to get subprocess coverage I have a coverage config. Actually I have a config file in all of them. Not really a big deal to add "patch=subprocess" when I need it. The way I thought about it is I don't want to enable a patch that I don't need - it's just extra complexity.

@nedbat @ionelmc Appreciate both of you taking the time to look at this.

At the scale of 1000s of repos, updating or creating in repo configs creates quite some friction.

I'd be happy to contribute a change to both coverage.py and pytest-cov to restore the previous default behavior as opt-in, with an option to opt out. Let me know if this direction aligns with what you both had in mind for subprocesses.