Problems with `combine` when not used from cmdline

[math-a3k]

Describe the bug The following snippet is being used for integrate coverage.py into Django's manage.py:

...
    if "test" in sys.argv:
        cov = Coverage()
        cov.erase()
        cov.start()

    execute_from_command_line(sys.argv)

    if "test" in sys.argv:
        cov.stop()
        cov.combine(strict=True)
        cov.save()
        covered = cov.report()
        if covered < 100:
            sys.exit(1)

and the following config:

[coverage:run]
source = .
omit =
  */node_modules/*
  */migrations/*
  */tests/*
  slackbotics/settings/*
  manage.py
branch = True
concurrency = multiprocessing
data_file = .coverage
relative_files = True
debug = config,dataio

[coverage:report]
show_missing = True
skip_covered = True
fail_under = 100
exclude_lines =
    pragma: no cover
    def __repr__
    def __str__
    if self.debug:
    if settings.DEBUG
    raise AssertionError
    raise NotImplementedError
    if 0:
    if __name__ == .__main__.:
    class .*\bProtocol\):
    @(abc\.)?abstractmethod

The problem is that when run from manage.py, it combines the files to the first one including the suffix and not renaming it to just .coverage. The report that shows is good, but when coverage html is run, it fails due to lack of data.

If coverage combine is ran from cmdline, it picks the left file and "renames" it to .coverage and everything works OK.

To Reproduce

1. What code shows the problem? Give us a specific commit of a specific repo that we can check

A repo with the code to reproduce is https://github.com/math-a3k/django_coverage_integration

Reproduction steps in README

Expected behavior combine invoked from control should behave the same than cmdline

Additional context I've debugging this for hours without any success, I found no difference in invocation or env yet. If you could provide any insight about what may be wrong, it will be highly appreciated.

Thanks!

[math-a3k]

This may not be a bug itself - as it works - but rather a question on how to integrate the software (sorry, I can't change the labels once submitted)

[nedbat]

I don't quite understand what's happening here yet, but I'm wondering why do it this way instead of using the coverage command-line interface?

[nedbat]

TBH, I don't quite understand why it behaves as it does, but you can fix it:

    if "test" in sys.argv:
        cov = Coverage()
        cov.erase()
        cov.start()

    execute_from_command_line(sys.argv)

    if "test" in sys.argv:
        cov.stop()
        cov = Coverage()          # <- add this line.
        cov.combine(strict=True)
        cov.save()
        covered = cov.report()
        if covered < 100:
            sys.exit(1)

[nedbat]

And this make some sense because when run from the command line, coverage run and coverage combine each start a new Coverage object.

[math-a3k]

Hi @nedbat! Thanks a lot!

[math-a3k]

@nedbat, it still doesn't report the coverage correctly. This would be to reproduce the behavior of pytest - where just by running "pytest" you get the coverage report once integration with coverage.py is done - in django test runner. This would be a nice to have, but a simple shell script:

#!/bin/sh

# Script intended to be easier with the coverage commands
# i.e. './coverage.sh' or './coverage.sh html'

set -e

coverage erase
coverage run manage.py test
coverage combine
coverage ${1:-"report"}

does the trick.

The idea was taken from https://adamj.eu/tech/2019/04/30/getting-a-django-application-to-100-percent-coverage/ (and Adam's book "Speed Up your Django Tests").

In a real codebase, I'm seeing a report of 15% in the "integrated" way and 48% in the "script" way. Leaving this here JIC someone comes up with the same.