Unintuitive behaviour when only subpackage provides py.typed

[Ranfir]

Bug Report

from numba import typed causes mypy to generate spurious attr-defined errors for everything else imported from numba.

To Reproduce

from numba import njit, typed  # error: Module has no attribute "njit"  [attr-defined]

Expected Behavior

scratch.py:1: error: Skipping analyzing "numba": module is installed, but missing library stubs or py.typed marker  [import]
scratch.py:1: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports

Actual Behavior

scratch.py:1: error: Module "numba" has no attribute "njit"  [attr-defined]

Your Environment

• Mypy version used: 1.5.1
• Mypy command-line flags: none
• Mypy configuration options from mypy.ini (and other config files): none
• Python version used: 3.8.8, 3.10.9, 3.11.5

$ .venv/bin/pip freeze
importlib-metadata==6.8.0
llvmlite==0.40.1
mypy==1.5.1
mypy-extensions==1.0.0
numba==0.57.1
numpy==1.24.4
tomli==2.0.1
typing_extensions==4.8.0
zipp==3.17.0

[LukeSavefrogs]

The same is happening with pydantic.

Example

Take the following snippet as example:

import pydantic

class CustomModel(pydantic.BaseModel):
    """Base class for pydantic models."""
    prop1: str = pydantic.Field(alias="property1", frozen=True)

This code will produce the following mypy errors:

• Line 3: Name "pydantic.BaseModel" is not defined [name-defined].
• Line 5: Module has no attribute "Field" [attr-defined].

[hauntsaninja]

This is because the numba.typed subpackage provides a py.typed file. As per PEP 561, I think this makes it look like a part of a typed namespace package that only provides that subpackage (see https://peps.python.org/pep-0561/#stub-only-packages). I think this is behaving as per the spec?

I don't know what LukeSavefrogs is about, I cannot reproduce nor have you provided enough information to truly reproduce.

[LukeSavefrogs]

I don't know what LukeSavefrogs is about, I cannot reproduce nor have you provided enough information to truly reproduce.

Sorry if I was not clear, when I come back home I'll try and see if the issue is related to what you said. Otherwise I'll open a new issue. 😄

[Ranfir]

This is because the numba.typed subpackage provides a py.typed file. As per PEP 561, I think this makes it look like a part of a typed namespace package that only provides that subpackage (see https://peps.python.org/pep-0561/#stub-only-packages). I think this is behaving as per the spec?

Thanks. I hadn't noticed the py.typed file in the numba.typed sub-package.

I don't think numba.typed qualifies as a namespace package in this case. My understanding per PEP 561 is that a sub-package can include a py.typed file to indicate that it and its sub-packages support typing. But in this case, it seems that mypy is seeing the py.typed file in the numba.typed sub-package and applying it to the entire numba package. To demonstrate this, note that omitting typed from the import results in the expected behavior:

from numba import njit

Package maintainers who wish to support type checking of their code MUST add a marker file named py.typed to their package supporting typing. This marker applies recursively: if a top-level package includes it, all its sub-packages MUST support type checking as well.

[LukeSavefrogs]

I don't know what LukeSavefrogs is about, I cannot reproduce nor have you provided enough information to truly reproduce.

Sorry if I was not clear, when I come back home I'll try and see if the issue is related to what you said. Otherwise I'll open a new issue. 😄

As an addition, I can confirm that pydantic too has a py.typed file

[c-gohlke]

is there any solution/workaround to this?

I can add

[tool.mypy]
disable_error_code = "attr-defined"

to pyproject.toml, but I would rather this happens at a module level

[tool.mypy-numba]
disable_error_code = "attr-defined"

does not work because the error happens in my modules at import time

[sumkincpp]

In my case, I cannot simply disable error by code for whole module (readline), though disabling globally in mypy works

[mypy]
strict = True

; # beware of enabling option, it can lead to false positives
; disable_error_code = attr-defined

; this doesn't work xD
[mypy-readline]
disable_error_code = attr-defined

[mypy-readline.*]
disable_error_code = attr-defined

[danielhundhausen]

Is there any update on this or recommended way of dealing with it?

[hauntsaninja]

I don't know about recommended, but here's one possible workaround:

[[tool.mypy.overrides]]
module = "numba.*"
follow_imports = "skip"

It will make mypy simply skip analysis of numba and anything originating from it will be Any.

[brandonchinn178]

+1 types-protobuf has the same issue, where

import google.protobuf.descriptor

errors with "Library google not found"

https://github.com/python/typeshed/issues/10986#issuecomment-2779287693

https://github.com/nipunn1313/mypy-protobuf/issues/681

[wyattscarpenter]

I'm not sure if this bug has subsequently been fixed. Assuming it hasn't, another workaround that might help is --follow-untyped-imports. This will try to analyze all the files. They probably won't get analyzed very well, but this is probably a bit more type safe than just skipping all analysis, if it works in one's particular case (it will probably also skip a lot of analysis, effectively, but it will do what it can).

[BryceBeagle]

This is still an issue (at least through mypy==1.18.2). For the specific issue I'm dealing with*, using follow-untyped-imports does indeed ~bypass the problem.

But then it explodes with a ton of other errors as you suggest might happen.

*mypy-protobuf generated code + types-protobuf stubs. See: https://github.com/nipunn1313/mypy-protobuf/issues/681

[BryceBeagle]

Actually, looking at this again:

I had initially tried using

[mypy]
follow_untyped_imports = True

but doing just

[mypy-google.protobuf]
follow_untyped_imports = True

seems to be just the right about of granularity. The imports I care about are no longer Any, but no other random errors in subpackages get surfaced.