[Bug]: Test failures on `2.2.0` — probably coverage related

[TheChymera]

What happened?

I get 18 test failures (21 if coverage is not installed) for =pynwb-2.2.0, even though I attempt to disable coverage: https://ppb.chymera.eu/4db145.log

Steps to Reproduce

This is the pytest invocation command (also seen in the log, everything, including the build is in the log linked above):

python3.10 -m pytest -vv -ra -l -Wdefault --color=yes -o console_output_style=count
    -p no:cov
    -p no:flake8
    -p no:flakes
    -p no:pylint
    -p no:markdown
    -p no:sugar
    -p no:xvfb

Traceback

This is one of the error tracebacks, all others are in the full log:

tests/validation/test_validate.py:166: AssertionError
_________________________________________ TestValidateCLI.test_validate_file_list_namespaces_core __________________________________________

self = <tests.validation.test_validate.TestValidateCLI testMethod=test_validate_file_list_namespaces_core>

    def test_validate_file_list_namespaces_core(self):
        """Test listing namespaces from a file"""
        result = subprocess.run(
            [
                "coverage", "run", "-p", "-m", "pynwb.validate", "tests/back_compat/1.1.2_nwbfile.nwb",
                "--list-namespaces"
            ],
            capture_output=True
        )

>       self.assertEqual(result.stderr.decode('utf-8'), '')
E       AssertionError: '/usr/lib/python3.10/site-packages/coverag[141 chars]")\n' != ''
E       - /usr/lib/python3.10/site-packages/coverage/control.py:801: CoverageWarning: No data was collected. (no-data-collected)
E       -   self._warn("No data was collected.", slug="no-data-collected")

result     = CompletedProcess(args=['coverage', 'run', '-p', '-m', 'pynwb.validate', 'tests/back_compat/1.1.2_nwbfile.nwb', '--list-namespaces'], returncode=0, stdout=b'core\n', stderr=b'/usr/lib/python3.10/site-packages/coverage/control.py:801: CoverageWarning: No data was collected. (no-data-collected)\n  self._warn("No data was collected.", slug="no-data-collected")\n')
self       = <tests.validation.test_validate.TestValidateCLI testMethod=test_validate_file_list_namespaces_core>

tests/validation/test_validate.py:178: AssertionError

Operating System

Linux

Python Executable

Python

Python Version

3.10

Code of Conduct

• [X] I agree to follow this project's Code of Conduct
• [X] Have you checked the Contributing document?
• [X] Have you ensured this bug was not already reported?

[oruebel]

@mavaylon1 could you take a look at this?

[TheChymera]

Deselected the failing tests for the time being on the Gentoo package, though it would really be best if they either (a) pass, or (b) — in case they are incompatible with the isolated testing (i.e. network) sandboxing — can be deselected dynamically. The problem with a static blacklist is it is bound to change from version to version and that delays new releases..

[TheChymera]

Issue still persists with =pynwb-2.5.0 and on Python 3.11

Full build log → https://ppb.chymera.eu/63ef58.log

[mavaylon1]

I will take a look at this

[rly]

Does this happen when you do python test.py? We haven't fully migrated to using pytest in pynwb, though we really should.

[mavaylon1]

I ran pytest -vv -ra -l -Wdefault --color=yes -o console_output_style=count locally with the latest pynwb and python 3.10 and it passes.

[TheChymera]

I think the issue is that coverage can't be disabled.

Here's what I get with

pytest -vv -ra -l -Wdefault -p no:cov --color=yes -o console_output_style=count

https://ppb.chymera.eu/043d81.log