[5.3.1] Paths in file behind COVERAGE_PROCESS_START need to be resolved relative to that file's location (rather than current working directory)?

[hartwork]

Hi!

I wouldn't know what to do without coveragepy so thanks for releasing and sharing this software as Open Source with the community. :pray:

I have been able to track this issue down finally and I believe it's a bug in coveragepy. Let's see what you think.

Describe the Bug

Setup and Symptom (overview)

• I have two Python files, one calling the other using subprocess
• I have properly setup sub-processes tracking a la https://coverage.readthedocs.io/en/coverage-5.1/subprocess.html
• I have relative paths in .coveragerc as is convenient and common
• The working directory of the two processes differ (and that's key)
• The report shows 100% for the caller but the callee file keeps showing 0% coverage.

I'll reproduce that setup from the ground up as a Bash session for you to reproduce further down.

Cause

The paths in the coverage RF file pointed to by environment variable COVERAGE_PROCESS_START are currently resolved relative to the current working directory. So two processes with different working directiories do not agree on eventual file locations (as proven below). Putting an absolute path into COVERAGE_PROCESS_START a la COVERAGE_PROCESS_START="${PWD}"/.coveragerc does not change anything about the sitation.

Workaround

Once I make all paths absolute in .coveragerc things work as expected. However that's not feasible most of the time and not more than a workaround.

To Reproduce

What follows is a single Bash shell session. This is with Python 3.7 on Linux (but Python 3.9 is the same).

Let's create two simple Python programs, one saying hello, one calling the other.

$ cd "$(mktemp -d)"
$ echo $'#! /usr/bin/env python\nimport os, sys\nprint(f"Hello! (ARGV {sys.argv!r}, PWD {os.getcwd()!r})")' | tee callee.py
#! /usr/bin/env python
import os, sys
print(f"Hello! (ARGV {sys.argv!r}, PWD {os.getcwd()!r})")

$ echo $'#! /usr/bin/env python\nimport subprocess\nsubprocess.call(["sh", "-c", "cd /tmp && callee.py"])' | tee caller.py
#! /usr/bin/env python
import subprocess
subprocess.call(["sh", "-c", "cd /tmp && callee.py"])

$ chmod a+x caller.py callee.py
$ PATH="${PATH}:${PWD}" ./caller.py
Hello! (ARGV ['/tmp/tmp.hZ5VSTGIQT/callee.py'], PWD '/tmp')

My choice if /tmp is arbitrary here, all we need is some other directory.

Now let's add minimal coverage tracking with support for sub-processes as documented:

$ python3 -m venv venv
$ source venv/bin/activate
$ pip install coverage
$ echo $'try:\n\timport coverage\n\tcoverage.process_startup()\nexcept ImportError:\n\tpass' | tee "$(ls -1d venv/lib/python*)"/site-packages/sitecustomize.py
try:
        import coverage
        coverage.process_startup()
except ImportError:
        pass

$ echo $'[run]\nconcurrency = multiprocessing\ndata_file = ./.coverage\nomit = ./venv/*\nsource = ./' | tee .coveragerc
[run]
concurrency = multiprocessing
data_file = ./.coverage
omit = ./venv/*
source = ./

$ coverage erase ; COVERAGE_PROCESS_START="${PWD}"/.coveragerc PATH="${PATH}:${PWD}" coverage run ./caller.py ; coverage combine ; coverage report
Hello! (ARGV ['/tmp/tmp.hZ5VSTGIQT/callee.py'], PWD '/tmp')
Name        Stmts   Miss  Cover
-------------------------------
callee.py       2      2     0%  <-- !!!
caller.py       2      0   100%
-------------------------------
TOTAL           4      2    50%

What's to note here is that callee.py has a coverage of 0% reported while I want to see 100% here.

Let's add some selective debugging to see what's happening:

$ coverage erase ; COVERAGE_DEBUG=config,pid,process,trace COVERAGE_PROCESS_START="${PWD}"/.coveragerc PATH="${PATH}:${PWD}" coverage run ./caller.py |& grep -E 'source:|run_omit:|Matcher|cwd is|config_files_read|data_file|callee\.py' ; ls -l {,/tmp/}.coverage.* ; coverage combine ; coverage report ; rm -fv {,/tmp/}.coverage.*
18734.d7ff: cwd is now '/tmp/tmp.hZ5VSTGIQT'
18734.d7ff: Source matching against trees <TreeMatcher ['/tmp/tmp.hZ5VSTGIQT']>
18734.d7ff: Omit matching: <FnmatchMatcher ['/tmp/tmp.hZ5VSTGIQT/venv/*']>
18734.d7ff:               config_files_read: /tmp/tmp.hZ5VSTGIQT/.coveragerc
18734.d7ff:                       data_file: ./.coverage
18734.d7ff:                        run_omit: ./venv/*
18734.d7ff:                          source: ./
18734.d7ff: Source matching against trees <TreeMatcher ['/tmp/tmp.hZ5VSTGIQT']>
18734.d7ff: Omit matching: <FnmatchMatcher ['/tmp/tmp.hZ5VSTGIQT/venv/*']>
18734.d7ff:               config_files_read: /tmp/tmp.hZ5VSTGIQT/.coveragerc
18734.d7ff:                       data_file: ./.coverage
18734.d7ff:                        run_omit: ./venv/*
18734.d7ff:                          source: ./
18736.411c: cwd is now '/tmp'
18736.411c: New process: cmd: ['/tmp/tmp.hZ5VSTGIQT/callee.py']
18736.411c: Source matching against trees <TreeMatcher ['/tmp']>                         <-- !!!
18736.411c: Omit matching: <FnmatchMatcher ['/tmp/venv/*']>                              <-- !!!
18736.411c:               config_files_read: /tmp/tmp.hZ5VSTGIQT/.coveragerc
18736.411c:                       data_file: ./.coverage                                 <-- !!!
18736.411c:                        run_omit: ./venv/*
18736.411c:                          source: ./
18736.411c: Tracing '/tmp/tmp.hZ5VSTGIQT/callee.py'
Hello! (ARGV ['/tmp/tmp.hZ5VSTGIQT/callee.py'], PWD '/tmp')
-rw-r--r-- 1 user123 user123 53248 Jan  6 15:51 .coverage.hostname123.18734.675103
-rw-r--r-- 1 user123 user123 53248 Jan  6 15:51 /tmp/.coverage.hostname123.18736.622739  <-- !!!
Name        Stmts   Miss  Cover
-------------------------------
callee.py       2      2     0%  <-- !!!
caller.py       2      0   100%
-------------------------------
TOTAL           4      2    50%
removed '/tmp/.coverage.hostname123.18736.622739'

So what's happening is:

• The paths in .coveragerc are resolved relative to the current working directory.
• The two processes have different working directories
• One process ends up writing to /tmp/tmp.hZ5VSTGIQT/coverage.hostname123.18734.675103, the other to /tmp/.coverage.hostname123.18736.622739.

Let me demo a workaround by making all paths absolute in .coveragerc ourselves. The sed-line below only works because I made sure that all paths start with a ./ prefix explicitly, earlier.

$ sed "s,\./,${PWD}/,g" -i .coveragerc  # make all relative paths absolute, -i requires GNU sed if you're on macOS
$ cat .coveragerc
[run]
concurrency = multiprocessing
data_file = /tmp/tmp.hZ5VSTGIQT/.coverage
omit = /tmp/tmp.hZ5VSTGIQT/venv/*
source = /tmp/tmp.hZ5VSTGIQT/

$ coverage erase ; COVERAGE_PROCESS_START="${PWD}"/.coveragerc PATH="${PATH}:${PWD}" coverage run ./caller.py ; coverage combine ; coverage report
Hello! (ARGV ['/tmp/tmp.hZ5VSTGIQT/callee.py'], PWD '/tmp')
Name        Stmts   Miss  Cover
-------------------------------
callee.py       2      0   100%  <-- !!!
caller.py       2      0   100%
-------------------------------
TOTAL           4      0   100%

Expected behavior

I hope that by now I have convinced you that relative paths in the covaragerc file pointed to by environment variable COVERAGE_PROCESS_START should be resolved relative to the location ofn the file, not the current working directory.

Thanks for your time and attention, best, Sebastian