coveragepy
coveragepy copied to clipboard
Published
20 hours ago •
nedbat
nedbat
Error when using concurrency on a ThreadingHTTPServer
Describe the bug I get an error instead of a report when running coverage on the following code.
To Reproduce How can we reproduce the problem? Please be specific. Don't link to a failing CI job. Answer the questions below:
-
What version of Python are you using? Python 3.8.10
-
What version of coverage.py shows the problem? The output of
coverage debug sysis helpful.
-- sys -------------------------------------------------------
coverage_version: 6.5.0
coverage_module: /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/coverage/__init__.py
tracer: -none-
CTracer: available
plugins.file_tracers: -none-
plugins.configurers: -none-
plugins.context_switchers: -none-
configs_attempted: .coveragerc
configs_read: /home/user.name/python-tests/tests/.coveragerc
config_file: /home/user.name/python-tests/tests/.coveragerc
config_contents: b'[run]\nparallel = True\nbranch = True\nconcurrency = multiprocessing,thread\ndynamic_context = test_function\n\n[html]\nshow_contexts = True'
data_file: -none-
python: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0]
platform: Linux-5.15.0-56-generic-x86_64-with-glibc2.29
implementation: CPython
executable: /home/user.name/.pyVEnv/py3.8_generic/bin/python
def_encoding: utf-8
fs_encoding: utf-8
pid: 383407
cwd: /home/user.name/python-tests/tests
path: /home/user.name/.pyVEnv/py3.8_generic/bin
/usr/lib/python38.zip
/usr/lib/python3.8
/usr/lib/python3.8/lib-dynload
/home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages
/home/user.name/.local/lib/python3.8/site-packages
/usr/lib/python3.8/site-packages
/usr/local/lib/python3.8/dist-packages
/usr/lib/python3/dist-packages
environment: FDZ_PYPI_TOKEN = ***
FDZ_PYPI_USERNAME = __token__
HOME = /home/user.name
command_line: /home/user.name/.pyVEnv/py3.8_generic/bin/coverage debug sys
sqlite3_version: 2.6.0
sqlite3_sqlite_version: 3.31.1
sqlite3_temp_store: 0
sqlite3_compile_options: COMPILER=gcc-9.4.0, ENABLE_COLUMN_METADATA, ENABLE_DBSTAT_VTAB,
ENABLE_FTS3, ENABLE_FTS3_PARENTHESIS, ENABLE_FTS3_TOKENIZER, ENABLE_FTS4,
ENABLE_FTS5, ENABLE_JSON1, ENABLE_LOAD_EXTENSION, ENABLE_PREUPDATE_HOOK,
ENABLE_RTREE, ENABLE_SESSION, ENABLE_STMTVTAB, ENABLE_UNLOCK_NOTIFY,
ENABLE_UPDATE_DELETE_LIMIT, HAVE_ISNAN, LIKE_DOESNT_MATCH_BLOBS,
MAX_SCHEMA_RETRY=25, MAX_VARIABLE_NUMBER=250000, OMIT_LOOKASIDE,
SECURE_DELETE, SOUNDEX, TEMP_STORE=1, THREADSAFE=1, USE_URI
- What versions of what packages do you have installed? The output of
pip freezeis helpful.
anyio==3.6.2
appdirs==1.4.3
apturl==0.5.2
argon2-cffi==21.3.0
argon2-cffi-bindings==21.2.0
asttokens==2.1.0
attrs==22.1.0
autokey==0.95.10
Babel==2.11.0
backcall==0.2.0
bcrypt==3.1.7
beautifulsoup4==4.11.1
bleach==5.0.1
blinker==1.4
Brlapi==0.7.0
certifi==2019.11.28
cffi==1.15.1
chardet==3.0.4
chrome-gnome-shell==0.0.0
Click==7.0
colorama==0.4.3
command-not-found==0.3
coverage==6.5.0
cryptography==2.8
cupshelpers==1.0
dbus-python==1.2.16
debugpy==1.6.3
decorator==5.1.1
defer==1.0.6
defusedxml==0.7.1
distlib==0.3.0
distro==1.4.0
distro-info===0.23ubuntu1
duplicity==0.8.12.0
entrypoints==0.3
exceptiongroup==1.0.4
executing==1.2.0
Faker==9.8.1
fasteners==0.14.1
fastjsonschema==2.16.2
filelock==3.0.12
future==0.18.2
galternatives==1.0.6
httplib2==0.14.0
idna==2.8
importlib-metadata==5.0.0
importlib-resources==5.10.0
iniconfig==1.1.1
ipykernel==6.4.2
ipython==7.34.0
ipython-genutils==0.2.0
jedi==0.18.1
Jinja2==3.1.2
json5==0.9.10
jsonschema==4.17.0
jupyter-server==1.23.1
jupyter_client==7.4.5
jupyter_core==5.0.0
jupyterlab==3.5.0
jupyterlab-pygments==0.2.2
jupyterlab_server==2.16.3
keyring==18.0.1
language-selector==0.1
launchpadlib==1.10.13
lazr.restfulclient==0.14.2
lazr.uri==1.0.3
lockfile==0.12.2
louis==3.12.0
macaroonbakery==1.3.1
Mako==1.1.0
MarkupSafe==2.1.1
matplotlib-inline==0.1.6
meld==3.20.2
mistune==2.0.4
monotonic==1.5
more-itertools==4.2.0
nbclassic==0.4.8
nbclient==0.7.0
nbconvert==7.2.4
nbformat==5.7.0
nest-asyncio==1.5.6
netifaces==0.10.4
notebook==6.5.2
notebook_shim==0.2.2
oauthlib==3.1.0
olefile==0.46
packaging==21.3
pandocfilters==1.5.0
paramiko==2.6.0
parso==0.8.3
pexpect==4.6.0
pickleshare==0.7.5
Pillow==7.0.0
pipenv==11.9.0
pkgutil_resolve_name==1.3.10
platformdirs==2.5.3
pluggy==1.0.0
prometheus-client==0.15.0
prompt-toolkit==3.0.32
protobuf==3.6.1
psutil==5.9.4
ptyprocess==0.7.0
pure-eval==0.2.2
pycairo==1.16.2
pycparser==2.21
pycups==1.9.73
Pygments==2.13.0
PyGObject==3.36.0
pyinotify==0.9.6
PyJWT==1.7.1
pymacaroons==0.13.0
PyNaCl==1.3.0
pyparsing==3.0.9
pyRFC3339==1.1
pyrsistent==0.19.2
pytest==7.2.0
python-apt==2.0.0+ubuntu0.20.4.8
python-dateutil==2.8.2
python-debian===0.1.36ubuntu1
python-xlib==0.23
pytz==2019.3
pyxdg==0.26
PyYAML==5.4
pyzmq==24.0.1
reportlab==3.5.34
requests==2.22.0
requests-unixsocket==0.2.0
SecretStorage==2.3.1
Send2Trash==1.8.0
simplejson==3.16.0
six==1.14.0
sniffio==1.3.0
soupsieve==2.3.2.post1
stack-data==0.6.0
systemd-python==234
terminado==0.17.0
text-unidecode==1.3
tinycss2==1.2.1
tomli==2.0.1
tornado==6.2
traitlets==5.5.0
ubuntu-advantage-tools==27.12
ubuntu-drivers-common==0.0.0
ufw==0.36
unattended-upgrades==0.1
urllib3==1.25.8
usb-creator==0.3.7
virtualenv==20.0.17
virtualenv-clone==0.3.0
wadllib==1.3.3
wcwidth==0.2.5
webencodings==0.5.1
websocket-client==1.4.2
xkit==0.0.0
zipp==1.0.0
- What code shows the problem? Give us a specific commit of a specific repo that we can check out. If you've already worked around the problem, please provide a commit before that fix.
test_main.py
import os
import signal
import time
from contextlib import contextmanager
from functools import partial
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer, SimpleHTTPRequestHandler
from multiprocessing import Process
from threading import Thread
import coverage
@contextmanager
def http_server(host: str, port: int, directory: str):
server = ThreadingHTTPServer(
(host, port), partial(SimpleHTTPRequestHandler, directory=directory)
)
server_thread = Thread(target=server.serve_forever, name="http_server")
server_thread.start()
try:
yield
finally:
server.shutdown()
server_thread.join()
def test_main():
# setup
coverage.process_startup()
with http_server("127.0.0.1", 8087, "."):
# now you can use the web server
time.sleep(1)
# test
a = 1
assert a == 1
.coveragerc
[run]
parallel = True
branch = True
concurrency = multiprocessing,thread
dynamic_context = test_function
[html]
show_contexts = True
- What commands did you run?
export COVERAGE_PROCESS_START=$PWD/.coveragerc
coverage run -m pytest test_main.py
Expected behavior
Get a coverage report
What I got
Error message
===================================================================================================== test session starts =====================================================================================================
platform linux -- Python 3.8.10, pytest-7.2.0, pluggy-1.0.0
rootdir: /home/user.name/python-tests/tests
plugins: anyio-3.6.2, Faker-9.8.1
collected 1 item
test_main.py . [100%]
====================================================================================================== 1 passed in 1.08s ======================================================================================================
self._collectors:
<Collector at 0x7fb2e198fa60: CTracer>
<module> : /home/user.name/.pyVEnv/py3.8_generic/bin/coverage:8
main : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/coverage/cmdline.py:943
command_line : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/coverage/cmdline.py:659
do_run : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/coverage/cmdline.py:827
start : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/coverage/control.py:580
_init_for_start : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/coverage/control.py:487
__init__ : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/coverage/collector.py:114
<Collector at 0x7fb2e021e940: CTracer>
<module> : /home/user.name/.pyVEnv/py3.8_generic/bin/coverage:8
main : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/coverage/cmdline.py:943
command_line : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/coverage/cmdline.py:659
do_run : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/coverage/cmdline.py:830
run : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/coverage/execfile.py:199
<module> : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/pytest/__main__.py:5
console_main : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/_pytest/config/__init__.py:190
main : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/_pytest/config/__init__.py:167
__call__ : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/pluggy/_hooks.py:265
_hookexec : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/pluggy/_manager.py:80
_multicall : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/pluggy/_callers.py:39
pytest_cmdline_main : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/_pytest/main.py:317
wrap_session : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/_pytest/main.py:270
_main : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/_pytest/main.py:324
__call__ : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/pluggy/_hooks.py:265
_hookexec : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/pluggy/_manager.py:80
_multicall : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/pluggy/_callers.py:39
pytest_runtestloop : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/_pytest/main.py:349
__call__ : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/pluggy/_hooks.py:265
_hookexec : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/pluggy/_manager.py:80
_multicall : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/pluggy/_callers.py:39
pytest_runtest_protocol : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/_pytest/runner.py:112
runtestprotocol : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/_pytest/runner.py:131
call_and_report : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/_pytest/runner.py:220
call_runtest_hook : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/_pytest/runner.py:259
from_call : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/_pytest/runner.py:339
<lambda> : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/_pytest/runner.py:260
__call__ : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/pluggy/_hooks.py:265
_hookexec : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/pluggy/_manager.py:80
_multicall : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/pluggy/_callers.py:39
pytest_runtest_call : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/_pytest/runner.py:167
runtest : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/_pytest/python.py:1789
__call__ : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/pluggy/_hooks.py:265
_hookexec : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/pluggy/_manager.py:80
_multicall : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/pluggy/_callers.py:39
pytest_pyfunc_call : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/_pytest/python.py:195
test_main : /home/user.name/python-tests/tests/test_main.py:34
process_startup : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/coverage/control.py:1223
start : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/coverage/control.py:580
_init_for_start : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/coverage/control.py:487
__init__ : /home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/coverage/collector.py:114
Traceback (most recent call last):
File "/home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/coverage/cmdline.py", line 830, in do_run
runner.run()
File "/home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/coverage/execfile.py", line 199, in run
exec(code, main_mod.__dict__)
File "/home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/pytest/__main__.py", line 5, in <module>
raise SystemExit(pytest.console_main())
SystemExit: ExitCode.OK
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/user.name/.pyVEnv/py3.8_generic/bin/coverage", line 8, in <module>
sys.exit(main())
File "/home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/coverage/cmdline.py", line 943, in main
status = CoverageScript().command_line(argv)
File "/home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/coverage/cmdline.py", line 659, in command_line
return self.do_run(options, args)
File "/home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/coverage/cmdline.py", line 835, in do_run
self.coverage.stop()
File "/home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/coverage/control.py", line 604, in stop
self._collector.stop()
File "/home/user.name/.pyVEnv/py3.8_generic/lib/python3.8/site-packages/coverage/collector.py", line 351, in stop
assert self._collectors[-1] is self, (
AssertionError: Expected current collector to be <Collector at 0x7fb2e198fa60: CTracer>, but it's <Collector at 0x7fb2e021e940: CTracer>
Just to give some context, the idea is to use requests to call to the server (a custom one, not the standard SimpleHTTP) instead of the
# test
a = 1
assert a == 1
And then check the coverage on that custom server. I get a different error with that code, but while creating a minimum example I got this error and thought I should start here.
@robs-nice99 Sorry I haven't gotten to this yet, but: it looks like you've exposed a secret in your FDZ_PYPI_TOKEN environment variable above. Sorry about that.
