pylint-pytest
pylint-pytest copied to clipboard
reverbc
[BUG]Why does adding a module produce F6401
Describe the bug
Can you explain the prompt 'F6401' when I run pylint pylint-pytest plugin cannot enumerate and collect pytest fixtures. Please run `pytest --fixtures --collect-only path/to/current/module.py` and resolve any potential syntax error or package dependency issues (Can-enumerate-pytest-fixtures) is the reason?
I would like to know how it works, or why it appears, and sometimes has different outputs. The same code, sometimes two, sometimes more. I was depressed.
I did run pytest --fixtures --collect-only without any unusual hints and my tests were normal.
Description:
After I fine-tune my existing code, including running pylint, pytest, and isort, everything works. I added a new package executor with three modules, one is the abstract module of base.py, two are corresponding to different implementation modules(local.py, docker.py).
Then I run isort, and pylint works fine
Then I import the base class and two implementation classes in the module's __init__.py file, and add a factory method.
When I run pylint again, the input tells me that some of the test modules have problems with F6401.
Again, I want to emphasize that everything was fine until I added this module. But now I just added the source code of this module, this exception will appear.
What makes it even more confusing to me is that the module I'm prompted doesn't include any fixtures. I ran pylint again and found that F6401 has more test modules (several times more than last time).
I've been using PyLint for a new project to check for a mode-by-module migration, and when I migrate to this module, I can't continue.
To Reproduce
Package versions
- pylint 3.0.0a3
- pylint-pytest 1.1.2
- pyparsing 2.4.7
- pytest 6.2.3
- pytest-asyncio 0.14.0
- pytest-cov 2.11.1
- pytest-mock 3.5.1
(add any relevant pylint/pytest plugin here)
load-plugins = pylint_pytest,
Folder structure
.
├── demo.py
├── dist
├── Pipfile
├── Pipfile.lock
├── README.md
├── setup.cfg
├── setup.py
├── src
│ ├── crawlerstack_spiderkeeper
│ │ ├── alembic
│ │ │ ├── alembic.ini
│ │ │ ├── env.py
│ │ │ ├── README
│ │ │ ├── script.py.mako
│ │ │ └── versions
│ │ │ ├── 17feeb1da886_init.py
│ │ │ ├── dae2b963c44f_.py
│ │ │ └── __init__.py
│ │ ├── cmdline.py
│ │ ├── config
│ │ │ ├── __init__.py
│ │ │ ├── settings.yaml
│ │ │ └── settings.yml
│ │ ├── dao
│ │ │ ├── artifact.py
│ │ │ ├── base.py
│ │ │ ├── __init__.py
│ │ │ ├── job.py
│ │ │ ├── project.py
│ │ │ ├── server.py
│ │ │ ├── storage.py
│ │ │ └── task.py
│ │ ├── db
│ │ │ ├── __init__.py
│ │ │ ├── models.py
│ │ ├── executor
│ │ │ ├── base.py
│ │ │ ├── docker.py
│ │ │ ├── __init__.py
│ │ │ ├── local.py
│ │ │ └── subprocess.py
│ │ ├── __init__.py
│ │ ├── schedule
│ │ │ ├── __init__.py
│ │ │ └── scheduler.py
│ │ ├── schemas
│ │ │ ├── artifact.py
│ │ │ ├── audit.py
│ │ │ ├── base.py
│ │ │ ├── __init__.py
│ │ │ ├── job.py
│ │ │ ├── project.py
│ │ │ ├── schedule.py
│ │ │ ├── server.py
│ │ │ ├── storage.py
│ │ │ └── task.py
│ │ ├── signals.py
│ │ └── utils
│ │ ├── constants.py
│ │ ├── exceptions.py
│ │ ├── __init__.py
│ │ ├── log.py
│ │ ├── metadata.py
│ │ ├── states.py
│ │ ├── types.py
│ │ └── virtualenv.py
│ └── crawlerstack_spiderkeeper.egg-info
│ ├── dependency_links.txt
│ ├── entry_points.txt
│ ├── not-zip-safe
│ ├── PKG-INFO
│ ├── requires.txt
│ ├── SOURCES.txt
│ └── top_level.txt
├── tests
│ ├── conftest.py
│ ├── dao
│ │ ├── __init__.py
│ │ ├── test_artifact.py
│ │ ├── test_base.py
│ │ ├── test_job.py
│ │ ├── test_project.py
│ │ ├── test_server.py
│ │ ├── test_storage.py
│ │ └── test_task.py
│ ├── data
│ │ ├── agpl-3.0.txt
│ │ └── demo
│ │ ├── demo
│ │ │ ├── extensions.py
│ │ │ ├── __init__.py
│ │ │ ├── items.py
│ │ │ ├── middlewares.py
│ │ │ ├── mock.py
│ │ │ ├── settings.py
│ │ │ └── spiders
│ │ │ ├── example.py
│ │ │ └── __init__.py
│ │ ├── Dockerfile
│ │ ├── Pipfile
│ │ ├── Pipfile.lock
│ │ ├── requirements.txt
│ │ ├── scrapy.cfg
│ │ └── setup.py
│ ├── __init__.py
│ ├── test_db.py
│ └── utils
│ ├── artifacts
│ │ ├── artifacts
│ │ ├── virtualenvs
│ │ └── workspaces
│ ├── __init__.py
│ ├── test_log.py
│ ├── test_metadata.py
│ └── tests.py
└── tox.ini
File content :
When I start this code, I get F6401, and when I comment it out, everything is fine.
crawlerstack_spiderkeeper.executor::__init__.py
"""
Executor API
"""
from typing import Type
from crawlerstack_spiderkeeper.config import settings
from crawlerstack_spiderkeeper.executor.base import BaseExecutor
from crawlerstack_spiderkeeper.executor.docker import DockerExecutor
from crawlerstack_spiderkeeper.executor.local import LocalExecutor
from crawlerstack_spiderkeeper.utils.exceptions import SpiderkeeperError
def executor_factory() -> Type[BaseExecutor]:
"""
Executor factory.
Create executor from settings.EXECUTOR
:return:
"""
executor = str(settings.EXECUTOR).lower()
if executor == 'local':
executor_kls = LocalExecutor
elif executor == 'docker':
executor_kls = DockerExecutor
else:
raise SpiderkeeperError('Invalid executor.')
return executor_kls
__all__ = [
'DockerExecutor',
'LocalExecutor',
'executor_factory'
]
pylint output with the plugin :
Why is it different before and after?
❯ pylint src tests
************* Module tests.dao.test_base
tests/dao/test_base.py:1:0: F6401: pylint-pytest plugin cannot enumerate and collect pytest fixtures. Please run `pytest --fixtures --collect-only path/to/current/module.py` and resolve any potential syntax error or package dependency issues (cannot-enumerate-pytest-fixtures)
--------------------------------------------------------------------
Your code has been rated at 10.00/10 (previous run: 10.00/10, +0.00)
❯
❯ pylint src tests
************* Module tests.test_db
tests/test_db.py:1:0: F6401: pylint-pytest plugin cannot enumerate and collect pytest fixtures. Please run `pytest --fixtures --collect-only path/to/current/module.py` and resolve any potential syntax error or package dependency issues (cannot-enumerate-pytest-fixtures)
************* Module tests.utils.test_metadata
tests/utils/test_metadata.py:1:0: F6401: pylint-pytest plugin cannot enumerate and collect pytest fixtures. Please run `pytest --fixtures --collect-only path/to/current/module.py` and resolve any potential syntax error or package dependency issues (cannot-enumerate-pytest-fixtures)
************* Module tests.utils.test_log
tests/utils/test_log.py:1:0: F6401: pylint-pytest plugin cannot enumerate and collect pytest fixtures. Please run `pytest --fixtures --collect-only path/to/current/module.py` and resolve any potential syntax error or package dependency issues (cannot-enumerate-pytest-fixtures)
************* Module tests.dao.test_task
tests/dao/test_task.py:1:0: F6401: pylint-pytest plugin cannot enumerate and collect pytest fixtures. Please run `pytest --fixtures --collect-only path/to/current/module.py` and resolve any potential syntax error or package dependency issues (cannot-enumerate-pytest-fixtures)
************* Module tests.dao.test_artifact
tests/dao/test_artifact.py:1:0: F6401: pylint-pytest plugin cannot enumerate and collect pytest fixtures. Please run `pytest --fixtures --collect-only path/to/current/module.py` and resolve any potential syntax error or package dependency issues (cannot-enumerate-pytest-fixtures)
************* Module tests.dao.test_job
tests/dao/test_job.py:1:0: F6401: pylint-pytest plugin cannot enumerate and collect pytest fixtures. Please run `pytest --fixtures --collect-only path/to/current/module.py` and resolve any potential syntax error or package dependency issues (cannot-enumerate-pytest-fixtures)
************* Module tests.dao.test_server
tests/dao/test_server.py:1:0: F6401: pylint-pytest plugin cannot enumerate and collect pytest fixtures. Please run `pytest --fixtures --collect-only path/to/current/module.py` and resolve any potential syntax error or package dependency issues (cannot-enumerate-pytest-fixtures)
************* Module tests.dao.test_project
tests/dao/test_project.py:1:0: F6401: pylint-pytest plugin cannot enumerate and collect pytest fixtures. Please run `pytest --fixtures --collect-only path/to/current/module.py` and resolve any potential syntax error or package dependency issues (cannot-enumerate-pytest-fixtures)
************* Module tests.dao.test_storage
tests/dao/test_storage.py:1:0: F6401: pylint-pytest plugin cannot enumerate and collect pytest fixtures. Please run `pytest --fixtures --collect-only path/to/current/module.py` and resolve any potential syntax error or package dependency issues (cannot-enumerate-pytest-fixtures)
************* Module tests.dao.test_base
tests/dao/test_base.py:1:0: F6401: pylint-pytest plugin cannot enumerate and collect pytest fixtures. Please run `pytest --fixtures --collect-only path/to/current/module.py` and resolve any potential syntax error or package dependency issues (cannot-enumerate-pytest-fixtures)
--------------------------------------------------------------------
Your code has been rated at 10.00/10 (previous run: 10.00/10, +0.00)
(Optional) pytest output from fixture collection
❯ pytest --fixtures --collect-only
======================================================================================================================================================================== test session starts ========================================================================================================================================================================
platform linux -- Python 3.7.3, pytest-6.2.3, py-1.10.0, pluggy-0.13.1
rootdir: /home/kevin/workspaces/develop/python/crawlerstack/crawlerstack-spiderkeeper, configfile: setup.cfg, testpaths: tests
plugins: asyncio-0.14.0, cov-2.11.1, mock-3.5.1
collected 60 items
<Package tests>
<Module test_db.py>
<Function test_migrate>
<Function test_db>
<Package dao>
<Module test_artifact.py>
<Function test_get_artifact_from_job_id>
<Function test_get_project_of_artifacts[True]>
<Function test_get_project_of_artifacts[False]>
<Module test_base.py>
<Function test_get>
<Function test_get_not_exist>
<Function test_get_multi_not_exist>
<Function test_get_multi[None-None-gt]>
<Function test_get_multi[DESC-None-gt]>
<Function test_get_multi[ASC-None-lt]>
<Function test_get_multi[None-datetime-gt]>
<Function test_get_multi[None-foo-SpiderkeeperError]>
<Function test_create>
<Function test_update[True]>
<Function test_update[False]>
<Function test_update_by_id[True]>
<Function test_update_by_id[False]>
<Function test_delete>
<Function test_delete_by_id[True]>
<Function test_delete_by_id[False]>
<Function test_delete_by_id_obj_not_exist>
<Function test_count>
<Module test_job.py>
<Function test_job_state[True]>
<Function test_job_state[False]>
<Module test_project.py>
<Function test_create>
<Module test_server.py>
<Function test_get_server_by_job_id>
<Module test_storage.py>
<Function test_increase_storage_count>
<Function test_running_storage>
<Module test_task.py>
<Function test_get_running>
<Function test_count_running_task>
<Function test_increase_item_count>
<Package utils>
<Module test_log.py>
<Function test_verbose_formatter[True-verbose]>
<Function test_verbose_formatter[False-simple]>
<Function test_log_level[True-DEBUG-DEBUG]>
<Function test_log_level[True-INFO-DEBUG]>
<Function test_log_level[True-ERROR-DEBUG]>
<Function test_log_level[False-DEBUG-DEBUG]>
<Function test_log_level[False-INFO-INFO]>
<Module test_metadata.py>
<Function test>
<Function test_from_project>
<Module tests.py>
<Function test_run_in_executor>
<Function test_app_data[APPID-1-1-expect_value0]>
<Function test_app_data[app_id1-expect_value1]>
<Function test_common_query_params[None-None-None-None-expect_value0]>
<Function test_common_query_params[None-100-None-None-expect_value1]>
<Function test_common_query_params[90-None-None-None-expect_value2]>
<Function test_common_query_params[90-100-None-None-expect_value3]>
<Function test_common_query_params[50-100-None-None-expect_value4]>
<Function test_get_host_address>
<Function test_upload>
<Function test_init_app_id_from_str>
<Function test_init_app_id>
<Function test_app_id_eq>
<Function test_kill_proc_tree>
<Function test_staging_path>
<Function test_head>
<Function test_last>
<Function test_tail[2048-22]>
<Function test_tail[100-4]>
cache
Return a cache object that can persist state between testing sessions.
cache.get(key, default)
cache.set(key, value)
Keys must be ``/`` separated strings, where the first part is usually the
name of your plugin or application to avoid clashes with other cache users.
Values can be any object handled by the json stdlib module.
capsys
Enable text capturing of writes to ``sys.stdout`` and ``sys.stderr``.
The captured output is made available via ``capsys.readouterr()`` method
calls, which return a ``(out, err)`` namedtuple.
``out`` and ``err`` will be ``text`` objects.
capsysbinary
Enable bytes capturing of writes to ``sys.stdout`` and ``sys.stderr``.
The captured output is made available via ``capsysbinary.readouterr()``
method calls, which return a ``(out, err)`` namedtuple.
``out`` and ``err`` will be ``bytes`` objects.
capfd
Enable text capturing of writes to file descriptors ``1`` and ``2``.
The captured output is made available via ``capfd.readouterr()`` method
calls, which return a ``(out, err)`` namedtuple.
``out`` and ``err`` will be ``text`` objects.
capfdbinary
Enable bytes capturing of writes to file descriptors ``1`` and ``2``.
The captured output is made available via ``capfd.readouterr()`` method
calls, which return a ``(out, err)`` namedtuple.
``out`` and ``err`` will be ``byte`` objects.
doctest_namespace [session scope]
Fixture that returns a :py:class:`dict` that will be injected into the
namespace of doctests.
pytestconfig [session scope]
Session-scoped fixture that returns the :class:`_pytest.config.Config` object.
Example::
def test_foo(pytestconfig):
if pytestconfig.getoption("verbose") > 0:
...
record_property
Add extra properties to the calling test.
User properties become part of the test report and are available to the
configured reporters, like JUnit XML.
The fixture is callable with ``name, value``. The value is automatically
XML-encoded.
Example::
def test_function(record_property):
record_property("example_key", 1)
record_xml_attribute
Add extra xml attributes to the tag for the calling test.
The fixture is callable with ``name, value``. The value is
automatically XML-encoded.
record_testsuite_property [session scope]
Record a new ``<property>`` tag as child of the root ``<testsuite>``.
This is suitable to writing global information regarding the entire test
suite, and is compatible with ``xunit2`` JUnit family.
This is a ``session``-scoped fixture which is called with ``(name, value)``. Example:
.. code-block:: python
def test_foo(record_testsuite_property):
record_testsuite_property("ARCH", "PPC")
record_testsuite_property("STORAGE_TYPE", "CEPH")
``name`` must be a string, ``value`` will be converted to a string and properly xml-escaped.
.. warning::
Currently this fixture **does not work** with the
`pytest-xdist <https://github.com/pytest-dev/pytest-xdist>`__ plugin. See issue
`#7767 <https://github.com/pytest-dev/pytest/issues/7767>`__ for details.
caplog
Access and control log capturing.
Captured logs are available through the following properties/methods::
* caplog.messages -> list of format-interpolated log messages
* caplog.text -> string containing formatted log output
* caplog.records -> list of logging.LogRecord instances
* caplog.record_tuples -> list of (logger_name, level, message) tuples
* caplog.clear() -> clear captured records and formatted log output string
monkeypatch
A convenient fixture for monkey-patching.
The fixture provides these methods to modify objects, dictionaries or
os.environ::
monkeypatch.setattr(obj, name, value, raising=True)
monkeypatch.delattr(obj, name, raising=True)
monkeypatch.setitem(mapping, name, value)
monkeypatch.delitem(obj, name, raising=True)
monkeypatch.setenv(name, value, prepend=False)
monkeypatch.delenv(name, raising=True)
monkeypatch.syspath_prepend(path)
monkeypatch.chdir(path)
All modifications will be undone after the requesting test function or
fixture has finished. The ``raising`` parameter determines if a KeyError
or AttributeError will be raised if the set/deletion operation has no target.
recwarn
Return a :class:`WarningsRecorder` instance that records all warnings emitted by test functions.
See http://docs.python.org/library/warnings.html for information
on warning categories.
tmpdir_factory [session scope]
Return a :class:`_pytest.tmpdir.TempdirFactory` instance for the test session.
tmp_path_factory [session scope]
Return a :class:`_pytest.tmpdir.TempPathFactory` instance for the test session.
tmpdir
Return a temporary directory path object which is unique to each test
function invocation, created as a sub directory of the base temporary
directory.
By default, a new base temporary directory is created each test session,
and old bases are removed after 3 sessions, to aid in debugging. If
``--basetemp`` is used then it is cleared each session. See :ref:`base
temporary directory`.
The returned object is a `py.path.local`_ path object.
.. _`py.path.local`: https://py.readthedocs.io/en/latest/path.html
tmp_path
Return a temporary directory path object which is unique to each test
function invocation, created as a sub directory of the base temporary
directory.
By default, a new base temporary directory is created each test session,
and old bases are removed after 3 sessions, to aid in debugging. If
``--basetemp`` is used then it is cleared each session. See :ref:`base
temporary directory`.
The returned object is a :class:`pathlib.Path` object.
------------------------------------------------------------------------------------------------------------------------------------------------------------ fixtures defined from pytest_asyncio.plugin ------------------------------------------------------------------------------------------------------------------------------------------------------------
event_loop
Create an instance of the default event loop for each test case.
unused_tcp_port
/home/kevin/.local/share/virtualenvs/crawlerstack-spiderkeeper-wbKs1mAt/lib/python3.7/site-packages/pytest_asyncio/plugin.py:221: no docstring available
unused_tcp_port_factory
A factory function, producing different unused TCP ports.
-------------------------------------------------------------------------------------------------------------------------------------------------------------- fixtures defined from pytest_cov.plugin --------------------------------------------------------------------------------------------------------------------------------------------------------------
no_cover
A pytest fixture to disable coverage.
cov
A pytest fixture to provide access to the underlying coverage object.
------------------------------------------------------------------------------------------------------------------------------------------------------------- fixtures defined from pytest_mock.plugin --------------------------------------------------------------------------------------------------------------------------------------------------------------
class_mocker [class scope]
Return an object that has the same interface to the `mock` module, but
takes care of automatically undoing all patches after each test method.
mocker
Return an object that has the same interface to the `mock` module, but
takes care of automatically undoing all patches after each test method.
module_mocker [module scope]
Return an object that has the same interface to the `mock` module, but
takes care of automatically undoing all patches after each test method.
package_mocker [package scope]
Return an object that has the same interface to the `mock` module, but
takes care of automatically undoing all patches after each test method.
session_mocker [session scope]
Return an object that has the same interface to the `mock` module, but
takes care of automatically undoing all patches after each test method.
--------------------------------------------------------------------------------------------------------------------------------------------------------------- fixtures defined from tests.conftest ----------------------------------------------------------------------------------------------------------------------------------------------------------------
session_factory [session scope]
Session factory fixture
uploaded_file
Uploaded file fixture.
migrate
migrate fixture.
session
Session fixture.
init_audit
Init audit fixture.
init_server
Init server fixture.
init_project
Init project fixture.
init_artifact
Init artifact fixture.
init_job
Init job fixture.
init_task
Init task fixture.
init_storage
Init storage fixture.
temp_dir
初始化测试目录，同时将测试目录赋值到 settings 上
因为测试中所有内容都会放在这个目录，所以在引用 settings 时除非没有引用这个 fixture
或者使用了这个 fixture 的其他 fixture，否则 settings.ARTIFACT_PATH 都将返回测试目录
:return:
base_dir
Base dir fixture.
data_dir
Test data dir fixture.
demo_zip
只用 test/data/demo 构建的测试数据
压缩文件中的目录结构
xxx-20191215152202.zip
.
|-- xxx
| |- xxx
| | |- spiders
| | | |- __init__.py
| | | └── xxx.py
| | |- settings.py
| | |- middlewares.py
| | └── items.py
| |- Dockerfile
| |- requirements.txt
| |- Pipfile
| |- Pipfile.lock
| |- setup.py
| |- setup.cfg
| |- scrapy.cfg
| └── README.md
-------------------------------------------------------------------------------------------------------------------------------------------------------------- fixtures defined from tests.utils.tests --------------------------------------------------------------------------------------------------------------------------------------------------------------
demo_file
Fixture demo file
tail
Fixture tail
------------------------------------------------------------------------------------------------------------------------------------------------------------- fixtures defined from tests.dao.test_base -------------------------------------------------------------------------------------------------------------------------------------------------------------
dao
dao fixture
==================================================================================================================================================================== 60 tests collected in 0.09s ====================================================================================================================================================================
Expected behavior
Everything is ok
Additional context Add any other context about the problem here.
After debugging the source code, I found out that the cause of my problems was an error in pylint-pytest when running pytest to collect fixtures from source code, and then pylint-pytest passed the error to PyLint.
My source code had a type annotation error that caused pytest to look for a fixture from that module that was wrong, and the error was passed to pylint. But why there is a different output is not clear to me.
From debugging the source code, we know that pylint-pytest registers itself with pylint, and when pylint checks all files, it passes the files to pylint-pytest's FixtureChecker.
https://github.com/reverbc/pylint-pytest/blob/62676386f80989cc0373d77bc5dc74acc635fd7a/pylint_pytest/checkers/fixture.py#L92-L142
The visit_module method in the FixtureChecker passes the file to pytest, running pytest <module_file> --fixtures --collect-only, At the same time load the FixtureCollector plug-in into pytest.
https://github.com/reverbc/pylint-pytest/blob/62676386f80989cc0373d77bc5dc74acc635fd7a/pylint_pytest/checkers/fixture.py#L125-L131
In pytest_collectreport , if an error is reported by pytest, it is logged and the error information is passed to pytest.
https://github.com/reverbc/pylint-pytest/blob/62676386f80989cc0373d77bc5dc74acc635fd7a/pylint_pytest/checkers/fixture.py#L24-L34
I don't think this logic makes sense. Pytest should only collect fixtures from the test modules, and instead of collecting fixtures from all modules, Pylint-Pytest should filter out the source code when PyLint checks.
Hope to fix this problem.
I'm seeing this today as well.
Curious about this statement:
Pytest should only collect fixtures from the test modules, and instead of collecting fixtures from all modules, Pylint-Pytest should filter out the source code when PyLint checks.
That makes sense to me, because I don't think I'd want to put fixtures in source code, but how would pylint (and subsequently pytest) know the difference between source and test code?
Hi @miketheman
In pylint-pytest logic, pylint-pytest or pytest does not know the difference between the source code and the test code. Because pylint checks all files, for example pylint src tests checks all python files in both directories. So pylint-pytest simply passes the file to
pytest that pytest needs to check and lets pytest collect fixtures in that file.
If aone source file is bad small, pytest will error.I had these problems because one of the methods in the source code had the wrong return value type. However, it should be indicated exactly by pylint, but the information above obviously doesn't pinpoint exactly where the problem is and what caused it. It just tells you that the loading fixture is wrong. And it has nothing to do with the wrong file.
If we only talk about pytest, pytest will load its own configuration before running, such as from pytest.ini or setup.cfg, depending on pytest itself. For example, I configured it in setup.cfg
[tool:pytest]
testpaths = tests
python_files = tests.py test_*.py
Then pytest will only look for the test code in the tests directory. It also filters the non-test source code files in the directory.
From this point on, if I fix this problem, I have a hypothesis:
Refer to pytest's filter logic for test files and filter out any logic that does not conform to the rules before passing the files to pytest for inspection.
Since I was having conflicts with dynaconf when using the pylint plugin pylint-django, I lowered my detection requirements and switched to the less stringent flake8 detection code. And that's just one of my current alternatives.
@whg517 Thanks for the excellent explanation! This appears similar to #16 and #20 - so possibly if this is fixed, all three will be resolved.
Refer to pytest's filter logic for test files and filter out any logic that does not conform to the rules before passing the files to pytest for inspection.
I did some debugging and came to a similar conclusion: if there's a way for the plugin to skip non-pytest files altogether, it'd fix the problem. Maybe even w/o taping into _pytest this plugin could introduce a wildcard setting for including/excluding files?
P.S. In my case, the underlying collection error was caused by the fact that the project uses implicit namespaces and so feeding full file path to modules confuses Python when it tries to resolve relative imports (because it doesn't know what the root package is). It is easy to reproduce once you run the proper command (in this case python -m pytest --fixtures --collect-only octomachinery/github/models/events.py):
=========================================================================== ERRORS ===========================================================================
___________________________________________________ ERROR collecting octomachinery/github/models/events.py ___________________________________________________
octomachinery/github/models/events.py:17: in <module>
from ...utils.asynctools import aio_gather
E ImportError: attempted relative import beyond top-level package
Any = typing.Any
Iterable = typing.Iterable
Mapping = typing.Mapping
TYPE_CHECKING = False
TextIO = <class 'typing.TextIO'>
Type = typing.Type
Union = typing.Union
_GidgetHubEvent = <class 'gidgethub.sansio.Event'>
__builtins__ = <builtins>
__cached__ = '/home/runner/work/octomachinery/octomachinery/octomachinery/github/models/__pycache__/events.cpython-39.pyc'
__doc__ = 'Generic GitHub event containers.' __file__ = '/home/runner/work/octomachinery/octomachinery/octomachinery/github/models/events.py'
__loader__ = <_frozen_importlib_external.SourceFileLoader object at 0x7ff8f7a0ab80>
__name__ = 'models.events'
__package__ = 'models'
__spec__ = ModuleSpec(name='models.events', loader=<_frozen_importlib_external.SourceFileLoader object at 0x7ff8f7a0ab80>, origin='/home/runner/work
/octomachinery/octomachinery/octomachinery/github/models/events.py')
annotations = _Feature((3, 7, 0, 'beta', 1), (3, 10, 0, 'alpha', 0), 16777216)
attr = <module 'attr' from '/home/runner/work/octomachinery/octomachinery/.tox/pre-commit/lib/python3.9/site-packages/attr/__init__.py'>
cast = <function cast at 0x7ff8f97e5430>
json = <module 'json' from '/opt/hostedtoolcache/Python/3.9.6/x64/lib/python3.9/json/__init__.py'>
pathlib = <module 'pathlib' from '/opt/hostedtoolcache/Python/3.9.6/x64/lib/python3.9/pathlib.py'>
uuid = <module 'uuid' from '/opt/hostedtoolcache/Python/3.9.6/x64/lib/python3.9/uuid.py'>
warnings = <module 'warnings' from '/opt/hostedtoolcache/Python/3.9.6/x64/lib/python3.9/warnings.py'>
___________________________________________________ ERROR collecting octomachinery/github/models/events.py ___________________________________________________
ImportError while importing test module '/home/runner/work/octomachinery/octomachinery/octomachinery/github/models/events.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/opt/hostedtoolcache/Python/3.9.6/x64/lib/python3.9/importlib/__init__.py:127: in import_module
return _bootstrap._gcd_import(name[level:], package, level)
octomachinery/github/models/events.py:17: in <module>
from ...utils.asynctools import aio_gather
E ImportError: attempted relative import beyond top-level package
================================================================== short test summary info ===================================================================
ERROR octomachinery/github/models/events.py - ImportError: attempted relative import beyond top-level package
ERROR octomachinery/github/models/events.py
=========================================================== no tests collected, 2 errors in 0.09s ============================================================
P.P.S. Another problem is that the error message gives an example with a dummy file path making it hard to debug. I think the message could be improved to specifically mention what file exactly causes the problem, instead of saying it's path/to/current/module.py.
This should fix it: https://github.com/reverbc/pylint-pytest/pull/22.
cc @reverbc @ssbarnea
Pytest should only collect fixtures from the test modules, and instead of collecting fixtures from all modules, Pylint-Pytest should filter out the source code when PyLint checks.
I believe we have other use cases that pytest should care about non-test modules.
Similar to #2 use case, test author can use separate modules (let's call them helper modules) to organize their fixtures/helpers if they have a large and complex test framework. In that case, if there's a coding error in one of the helper modules and causing the pytest failed to enumerate the fixtures, pylint-pytest will not be able to suppress the FP. That was the main intention when I introduced this new warning, and unfortunately the fixes in #22 will not be able to prevent that from happening since it'll ignore the collection errors from helper modules.
I'm now more inclined to temporary remove this cannot-enumerate-pytest-fixtures check if it cannot correctly identify which modules actually contain pytest fixtures until we have a better solution to detect that.
I do not think so. Checking the test modules should load dependencies in the background anyway. If they are loaded properly, they'll return the fixtures. If not, they'll cause errors.
The problem with pointing pytest at random files is that those may be loaded in different ways (runpy vs scripts) and depending on whether there's __init__.py the module loader may or may not recognize the package called tests, for example.
So I'm pretty sure that pointing at the test files will identify the missing fixtures one way or another.