mypy icon indicating copy to clipboard operation
mypy copied to clipboard

exclude should imply a per-module follow import skip for matching modules

Open jwodder opened this issue 4 years ago • 4 comments

Bug Report

I recently converted this project (linked to at the most recent commit) from a plain script to a Python package. This included adding an un-annotated _version.py file generated by versioneer to the package. mypy (when run as mypy src) now reports errors due to _version.py lacking any type annotations, and I cannot get it to ignore that file.

What I've tried:

  • Adding exclude = _version.py to the [mypy] section in setup.cfg
  • Adding exclude = src/tinuous/_version.py to the [mypy] section
  • Adding exclude = src/tinuous/_(version|_init__).py to the [mypy] section
  • Running just mypy src/tinuous/__main__.py
  • Running mypy --exclude _version.py src

In all of these cases, mypy spits out errors relating to _version.py even though I thought I told it to ignore that file.

Your Environment

  • Mypy version used: 0.812
  • Mypy command-line flags: --exclude or nothing
  • Mypy configuration options from mypy.ini (and other config files):
[mypy]
ignore_missing_imports = True
disallow_untyped_defs = True
disallow_incomplete_defs = True
no_implicit_optional = True
warn_redundant_casts = True
warn_return_any = True
warn_unreachable = True
local_partial_types = True
no_implicit_reexport = True
strict_equality = True
show_error_codes = True
show_traceback = True
pretty = True
plugins = pydantic.mypy
# Various "exclude" options (see above)
  • Python version used: 3.9.4
  • Operating system and version: macOS 11.2.1

jwodder avatar Apr 27 '21 21:04 jwodder

This is a pretty reasonable confusion / ask.

Here's some background:

  • Previously, mypy used __init__.py to figure out what directories are Python packages. However, this approach doesn't work with namespace packages (packages no longer need to have __init__.py in Python 3).
  • When we changed mypy's directory discovery logic to allow for namespace packages, it start finding things in various subtrees that mypy would previously ignore due to an absence of __init__.pys. --exclude was introduced as a way to ignore (newly) discovered files from being discovered.
  • However, --exclude currently only affects mypy's recursive file discovery logic (given a folder src, what files inside it should I check), and not its import following logic (src can import zillions of Python files from zillions of places, and mypy follows them all).
  • You can see the difference in practice by removing all imports of _version from your program and noticing that mypy won't check it, despite it being in src.

That's all to say adding something like the following (in addition to --exclude) should work:

[mypy-tinuous._version]
follow_imports = skip

I don't really expect users to understand all of that / seems reasonable to me that --exclude should do the equivalent of follow_imports=skip automatically. Might be a tiny bit tricky since one operates with module names and one operates with filenames and some subtleties to do with relative paths.

hauntsaninja avatar Apr 27 '21 22:04 hauntsaninja

@hauntsaninja That explains why just exclude = _version.py doesn't work, but it doesn't explain why exclude = src/tinuous/_(version|_init__).py and mypy src/tinuous/__main__.py don't work, as __main__.py doesn't import __init__.py or _version.py.

jwodder avatar Apr 27 '21 22:04 jwodder

I found this ticket after being driven crazy by the same issue. But I can't make your suggested solution work with mypy 0.812. I am trying to check ts_salobj which has an auto-generated version.py (not present in the git repo) that is built before mypy runs and is not compatible with mypy. I have tried every variant of your suggested fix and none of them work, including even disabling follow_imports globally (which is not how I want to run):

[mypy]
ignore_missing_imports = True
exclude = version.py  # or version\.py
follow_imports = skip

For the record, the line in version.py that makes mypy unhappy is:

__dependency_versions__ = {}

Any other suggestions for working around this issue? We can eventually fix the generated version.py, but not immediately and I would much rather not type-check that file.

r-owen avatar Jul 06 '21 16:07 r-owen

The solution I ended up with was two-fold:

  1. Avoid importing version.py when type checking:
import typing

if typing.TYPE_CHECKING:
    __version__ = "?"
else:
    try:
        from .version import *
    except ImportError:
        __version__ = "?"
  1. Block version.py in two places in my setup.cfg file:
[tool:pytest]
addopts = --flake8 --mypy --ignore-glob=*version.py
...
[mypy]
ignore_missing_imports = True
exclude = version\.py

The [tool:pytest] section is sufficient to make pytest happy, but the exclude in the [mypy] section is useful when running mypy directly from the command line.

r-owen avatar Jul 06 '21 18:07 r-owen