Support `.coveragerc.toml` for configuration

[OlenaYefymenko]

This patch provides a new configuration — .coveragerc.toml . Considering that many projects have switched to toml configurations, this change offers a more flexible approach to manage coverage settings.

Resolves #1643

[OlenaYefymenko]

@nedbat, I've opened PR with the improvements, but I have a few doubts about my implementation. I've added comments in the code. I'd really appreciate your feedback on whether this approach works well. Thank you!

[nedbat]

Thanks for doing this. There are some other changes needed. Let me know which you want to do, and which I should do:

• Update the documentation
• Add an entry to CHANGES.rst
• Add your name to CONTRIBUTORS.txt

[OlenaYefymenko]

Thanks for doing this. There are some other changes needed. Let me know which you want to do, and which I should do:

• Update the documentation
• Add an entry to CHANGES.rst
• Add your name to CONTRIBUTORS.txt

Yes, I will add the changes

[OlenaYefymenko]

I'm in progress - some tests have failed, I'm going to fix them

[ofek]

I'm very excited to start using this feature, thanks for the effort here @OlenaYefymenko!!!

[nedbat]

@webknjaz @OlenaYefymenko What is left to do with this PR?

[webknjaz]

@nedbat just the change log move. Olena was going to mark is as ready just about now.

[OlenaYefymenko]

@webknjaz @OlenaYefymenko What is left to do with this PR?

@nedbat the pull request is ready for review. Please let me know if anything needs to be edited. Thank you for your help and patience

[nedbat]

Thanks for doing this, and thanks for persisting.

With these changes, the .coverage.toml file would have entries like:

[tool.coverage.run]
branch = true

but wouldn't we want this?:

[run]
branch = true

I'd like to make things seem natural, and I also like to give people options, so it's hard to strike the right balance. When I read (for example) pytest's explanation of their configuration rules my head starts to spin.

[webknjaz]

but wouldn't we want this?:

[run]
branch = true

I've seen both startegies in the wild and I'm leaning towards not having differences with pyproject.toml.

For example, Towncrier has [tool.towncrier] in towncrier.toml and in cython-lint, I asked for the same [tool.cython-lint] in .cython-lint.toml. I also insisted on this when we were getting a PR in pip-tools — you'll have [tool.pip-tools] in .pip-tools.toml. PyLint does the same. Pyright seems to have implemented parsing custom .toml files as pyproject.toml (https://github.com/microsoft/pyright/pull/7997) but it's unclear if it picks up a .pyright.toml as the default automatically. Poetry has a poetry.toml for the tool configuration that isn't PEP 621 packaging metadata: https://python-poetry.org/docs/configuration/#local-configuration. For Black, it's a bit confusing but it mentions .black files in user-global locations + passing a custom name on the command line: https://black.readthedocs.io/en/stable/usage_and_configuration/the_basics.html#where-black-looks-for-the-file — they are likely TOML with the same structure too, but I haven't checked. isort doesn't really make it clear how it parses a custom config file: https://pycqa.github.io/isort/docs/configuration/config_files#custom-config-files.

I did mention this opinion in the original issue, FTR: #1643.

My reasoning for this is that it's easier to migrate/move things between pyproject.toml and a .{tool-name}.toml for the end-users, UX-wise. And it's more straightforward when there's no differences with custom filenames: what if one passes a --config=an-arbitrary-file-name-that-is-not-pyproject.toml — which way we'd want to parse it? If they are the same, the same parser is used and there's no confusion between adding or not adding section prefixes. But if there's differences, this would incur more maintenance burden for both project maintenance and end-users.

I'd like to make things seem natural, and I also like to give people options, so it's hard to strike the right balance. When I read (for example) pytest's explanation of their configuration rules my head starts to spin.

Right. I missed the PR it when was being implemented.. I'd ask for a prefix if I'd seen it, but there's no going back I guess.

Ruff has the section/prefix removed, OTOH: https://docs.astral.sh/ruff/configuration/#__tabbed_1_2. It does save a few keystrokes but I'd rather keep it uniform. Hatch also has unprefixed sections: https://hatch.pypa.io/latest/config/environment/overview/#environment-configuration.

[ofek]

Please, please do not support the tool.coverage prefix in this new file.

1. Not matching the structure of the existing .coveragerc dedicated config file violates the principle of least surprise.
2. It's unnatural for design decisions of individual projects to be influenced by the language's packaging configuration. For example, Cargo.toml files support the same type of table called package.metadata that is ignored and is meant for tools. I've never seen config for projects that happen to be written in Rust, or are for Rust development, use that prefix.
3. We should take note of what developers use the most rather than noting precedent for keeping the prefix. Here are examples of popular projects (some already mentioned) that support dedicated config files without a prefix: pytest, Ruff, uv, Hatch, tox, etc. It's worth noting as well that projects receiving more maintenance tend to favor this approach to decouple from historical packaging nuances and provide a better UX.
4. Tools that compute a dependency graph like uv cannot distinguish distribution-affecting pyproject.toml changes from others because it's all about the file hash (same as other build systems like Bazel). In this case, commands like uv run must rebuild the package whenever changes occur and it becomes advantageous to extract configuration that doesn't affect the build artifacts. This is particularly cumbersome for projects with extension modules: one change to coverage measurement means waiting dozens of seconds or more unnecessarily.

@OlenaYefymenko Would you be open to such a change in this PR?

[webknjaz]

That's up to @nedbat to decide, then.

[ofek]

Oh, for sure! I was just asking if @OlenaYefymenko would personally be fine with that approach.

[OlenaYefymenko]

Would you be open to such a change in this PR?

@ofek thank you for the detailed explanation. Yes, I will make the necessary changes. I'm just waiting for the final decision from @nedbat

[nedbat]

I hope this doesn't seem like a cop-out: I think there are good arguments for both side, and I'd like to support both prefixed and un-prefixed settings, aiming for the least surprise. The only complexity here is that in pyproject.toml, we need to insist on the prefix. We already have precedent for this.

In tomlconfig.py we have this:

prefixes = ["tool.coverage."]
for prefix in prefixes:
    real_section = prefix + section

This code looks silly (why iterate over a fixed one-element list?), but it's that way because config.py has this:

class HandyConfigParser(configparser.ConfigParser):
    def __init__(self, our_file: bool) -> None:
        """Create the HandyConfigParser.

        `our_file` is True if this config file is specifically for coverage,
        False if we are examining another config file (tox.ini, setup.cfg)
        for possible settings.
        """
        self.section_prefixes = ["coverage:"]
        if our_file:
            self.section_prefixes.append("")

That code is there because we insist on the "coverage:" prefix in tox.ini and setup.cfg, but are flexible in .coveragerc or in an explicitly specified config file.

It would take just adding two lines to tomlconfig.py to let it read either style of section, while being strict about the sections in pyproject.toml.

To avoid over-complicating the docs, I'd mention the flexibility of section names in the opening paragraphs about syntax, but in the examples, only show the fully prefixed syntax. Maybe the tab name could be just "toml"?

I hope everyone will be happy with this approach.

[webknjaz]

Sounds reasonable.

[ofek]

I think it would be good to mention the possibility of either naming in the opening but I disagree about showing the fully prefixed syntax in the examples because then that would encourage folks to copy/paste that forever. I would imagine that there would be a new tab that comes first here:

This new tab would have the name .coveragerc.toml as implemented in this PR and the second tab would be pyproject.toml. At this point I think two options would be reasonable for the INI-formatted files:

1. Maintain their tabs and add a note to discourage their use either in the title or in the content after clicking.
2. Extract them into a separate collapsible section for legacy files.

Making the new standalone TOML file the one that is shown by default as well as its syntax would then maintain complete compatibility with the current documentation that starts the sections with the name of the options and no prefix:

This would also match strategy of the pytest config docs placing what users are likely to prefer first on the page. I won't go into it here but there is a strong preference amongst developers for TOML, YAML, etc. over INI-style partly due to how uncommon it is nowadays and partly due to the ease of parsing such configuration with tooling written in other languages (never in the standard library).

In any case even if there is disagreement here about the preferred file format I think the prefix should not be prominent when, like I mentioned, it really is just an artifact of a packaging quirk in our specific language. I'm heavily involved in Python packaging now but I wasn't around when that was happening, so I'm sorry about that.

[webknjaz]

Agreed, a dedicated tab sounds good. That's sounds closer to what the original feature request was after.

[nedbat]

I appreciate the thought and effort you are putting into this. Four tabs sounds reasonable. I'd prefer this order, and I've snuck in a comment to help people understand the flexibility in the TOML syntax:

This patch does it:

diff --git a/doc/cog_helpers.py b/doc/cog_helpers.py
index 1ff4e1aa..be681074 100644
--- a/doc/cog_helpers.py
+++ b/doc/cog_helpers.py
@@ -76,7 +76,7 @@ def show_configs(ini, toml):
     The equivalence is checked for accuracy, and the process fails if there's
     a mismatch.

-    A three-tabbed box will be produced.
+    A four-tabbed box will be produced.
     """
     ini, ini_vals = _read_config(ini, "covrc")
     toml, toml_vals = _read_config(toml, "covrc.toml")
@@ -89,11 +89,15 @@ def show_configs(ini, toml):
             )

     ini2 = re.sub(r"(?m)^\[", "[coverage:", ini)
+    toml2 = "# You can also use sections like [tool.coverage.run]\n"
+    toml2 += toml.replace("[tool.coverage.", "[")
+
     print()
     print(".. tabs::\n")
     for name, syntax, text in [
+        ("pyproject.toml", "toml", toml),
         (".coveragerc", "ini", ini),
-        (".coveragerc.toml or pyproject.toml", "toml", toml),
+        (".coverage.toml", "toml", toml2),
         ("setup.cfg or tox.ini", "ini", ini2),
     ]:
         print(f"    .. code-tab:: {syntax}")

BTW, I hadn't really noticed until now, but this PR names the file .coveragerc.toml, when it should be .coverage.toml.

We can work on the exact wording of the opening syntax section when the time comes.

Also, it's impossible to account for all the syntax possibilities. TOML is flexible enough that either of these will work in pyproject.toml:

[tool.coverage.run]
branch = true
core = "sysmon"

[tool.coverage]
run = { branch = true, core = "sysmon" }

[webknjaz]

BTW, I hadn't really noticed until now, but this PR names the file .coveragerc.toml, when it should be .coverage.toml.

I requested that name originally since it sounded more familiar. The precedent is .pylintrc/.pylintrc.toml/pylintrc/pylintrc.toml.

Also, wouldn't .coverage.toml be confused with the .coverage SQLite db?

[nedbat]

BTW, I hadn't really noticed until now, but this PR names the file .coveragerc.toml, when it should be .coverage.toml.

I requested that name originally since it sounded more familiar. The precedent is .pylintrc/.pylintrc.toml/pylintrc/pylintrc.toml.

Also, wouldn't .coverage.toml be confused with the .coverage SQLite db?

Good points! .coveragerc.toml it is!

[serban]

This is slightly off topic, but since it came up:

I prefer not to hide configuration files in my repositories. In addition to .coveragerc.toml, it would be great to support the same functionality at coveragerc.toml, without the leading dot.

I'd be happy to take my bike shedding to a new issue so as to not derail this pull request. Thanks for your effort on this!

[webknjaz]

@serban many people prefer the opposite — not seeing many files in repo roots. That's why a lot of them almost religiously dump hundred of lines into pyproject.toml, I imagine. It may be a good idea to support both, it's a common convention. I'm not sure if we should grow the scope of the PR, though.

[AA-Turner]

BTW, I hadn't really noticed until now, but this PR names the file .coveragerc.toml, when it should be .coverage.toml.

I'm not sure I agree that .coverage.toml (config file) would be confused for .coverage (results DB). They have different filenames, and one is a plain text file. I'd also suggest that the 'rc' suffix can be unhelpful -- I had to google for what it meant, and it seems that there's no consensus answer!

Other config (.pytest.toml, .ruff.toml, pyrefly.toml, ty.toml, etc) just uses the project name, which I think is more familiar / less surprising when getting started.

A

[nedbat]

I'm not sure I agree that .coverage.toml (config file) would be confused for .coverage (results DB).

The problem isn't that people will confuse the two. The default database name is .coverage, and if you use parallel=true, then other databases are named .coverage.somethingxyz, and coverage combine will look for .coverage.* to combine. We could exclude *.toml from combining, but now it's getting fiddly.

The good news is that you can name your configuration file anything you want if you are willing to use a command-line option or environment variable to refer to it.

[ofek]

@OlenaYefymenko Heads up that a rebase is now required because there are conflicts.

[OlenaYefymenko]

@OlenaYefymenko Heads up that a rebase is now required because there are conflicts.

@ofek yep. I will do the rebase, yesterday I had some issues with it for some reason. In the next commit I will also add the test, and that will be the final update. Thanks for pointing it out.