"sqlite3.OperationalError: database is locked" on CIFS filesystem

[adamjstewart]

Summary

It seems that pytest-cov does not support the CIFS filesystem.

Expected vs actual result

I would expect the following to run without error:

$ pytest --cov=.

Instead, I see the following error after all tests are run and before any coverage metrics are reported:

INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/_pytest/main.py", line 269, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/_pytest/main.py", line 323, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/pluggy/hooks.py", line 286, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/pluggy/manager.py", line 93, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/pluggy/manager.py", line 84, in <lambda>
INTERNALERROR>     self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
INTERNALERROR>   File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/pluggy/callers.py", line 203, in _multicall
INTERNALERROR>     gen.send(outcome)
INTERNALERROR>   File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/pytest_cov/plugin.py", line 271, in pytest_runtestloop
INTERNALERROR>     self.cov_controller.finish()
INTERNALERROR>   File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/pytest_cov/engine.py", line 44, in ensure_topdir_wrapper
INTERNALERROR>     return meth(self, *args, **kwargs)
INTERNALERROR>   File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/pytest_cov/engine.py", line 230, in finish
INTERNALERROR>     self.cov.stop()
INTERNALERROR>   File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/coverage/control.py", line 659, in save
INTERNALERROR>     data = self.get_data()
INTERNALERROR>   File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/coverage/control.py", line 727, in get_data
INTERNALERROR>     if self._collector and self._collector.flush_data():
INTERNALERROR>   File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/coverage/collector.py", line 442, in flush_data
INTERNALERROR>     self.covdata.add_lines(self.mapped_file_dict(self.data))
INTERNALERROR>   File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/coverage/sqldata.py", line 438, in add_lines
INTERNALERROR>     self._choose_lines_or_arcs(lines=True)
INTERNALERROR>   File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/coverage/sqldata.py", line 495, in _choose_lines_or_arcs
INTERNALERROR>     with self._connect() as con:
INTERNALERROR>   File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/coverage/sqldata.py", line 300, in _connect
INTERNALERROR>     self._create_db()
INTERNALERROR>   File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/coverage/sqldata.py", line 250, in _create_db
INTERNALERROR>     db.executescript(SCHEMA)
INTERNALERROR>   File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/coverage/sqldata.py", line 1114, in executescript
INTERNALERROR>     self.con.executescript(script)
INTERNALERROR> sqlite3.OperationalError: database is locked

Reproducer

Run pytest --cov=. on any CIFS filesystem. I'm encountering this issue on Microsoft Azure.

Versions

$ python -V
Python 3.8.1
$ pip list | grep pytest
pytest                                  6.2.4
pytest-cov                              2.12.1

Config

[tool.pytest.ini_options]
norecursedirs = [
    "*.egg*",
    ".*",
    "build",
    "data",
    "dist",
    "docs",
    "__pycache__",
]

Code

Should be able to reproduce with any code that contains tests.

[GiorgioSgl]

I have same problem, have you solve in some way?

I launch pytest --cov= test.py python version is 3.7

[RonnyPfannschmidt]

the error is coming out from coverage itself

[RonnyPfannschmidt]

but my basic understanding is - that none of the network filesystems offer reliable locking as sqlite would need it

[edgBR]

Same issue here. What is the workaround?

[RonnyPfannschmidt]

nope, needs a upstream bug in coverage itself

[adamjstewart]

Does someone else want to open a bug with coverage? I no longer have access to the system where I first encountered this issue, so I can no longer help test fixes.

[FaridDelgadoTR]

upstream bug created: #594

[RonnyPfannschmidt]

Closing as upstream issue

[nedbat]

I'm confused: the "upstream issue" is in this same repo?

[RonnyPfannschmidt]

Same here As far as I can tell, Sqlite doesn't work on cifs

[FaridDelgadoTR]

My bad, I meant to create this issue in https://github.com/nedbat/coveragepy. Do you guys think this is the right place?

[nedbat]

That is probably the right place, but if the problem is that SQLite doesn't work on CIFS, then there's not much we're going to be able to do about it. Ideally, you'd have a simple way for us to reproduce it, and even better, an experiment with a simple SQLite db on CIFS, independent of coverage.py.

[FaridDelgadoTR]

I will close #594 and create an issue in coverage when I have a more practical / easier way to reproduce the issue. Thanks @nedbat.

[RosalynLP]

Hello is there any update on this or a linked issue in coverage?