mypy is failing on `from git import Blob`

[pbeza]

Why mypy is failing when running this code:

$ cat /tmp/test.py 
from git import Blob

$ mypy /tmp/test.py
/tmp/test.py:1: error: Module "git" has no attribute "Blob"
Found 1 error in 1 file (checked 1 source file)

$ python3 -c "import git; print(git.__version__)"
3.1.23

$ mypy --version
mypy 0.910

$ python3 --version
Python 3.9.7

?

I can see that there is a conditional import in git/objects/base.py:

if TYPE_CHECKING:
    from git.repo import Repo
    from gitdb.base import OStream
    from .tree import Tree
    from .blob import Blob
    from .submodule.base import Submodule
    from git.refs.reference import 

and it seems that TYPE_CHECKING is set to False (I guess so based on the above error). Do you know why is that? How to workaround the problem?

[Byron]

Thanks for letting us know. If Blob was previously available and now it is behind a conditional, this may be an issue that was recently introduced.

However, it looks like it should rather be imported from git.objects, and maybe that was also the case historically.

[Yobmod]

Initial thoughts are that the import from .blob is a relative import in the objects module, and the if TYPE_CHECKING is true for mypy, so I dont think that is the problem. mypy is working on all those files in CI

What is test.py? Your own code? Can I see it?

I do know mypy doesn't work on our test files, because it doesn't like imports like from git import Blob, where Blob is defined in a submodule (then imported in the parent module as a * import) It wants the import to be from the file Blob is defined in (or explicitly exported from) like from git.objects.blob import Blob or from git.objects.base import Blob. (Note, python code itself works, just mypy thinks it is not safe to reexport * imports)

You might just be able to change a mypy setting to get it to pass. E.g allow_reexport = True and strict = False (see the mypy settings in our pyproject.toml.) if using strict, the order of settings also makes a difference).

If changing the import or mypy settings fixes it, let me know. Replacing all * imports is on the todo list for futures release, but will take a while, so I'll move it to its own issue if that's the problem here.

[pbeza]

What is test.py? Your own code? Can I see it?

This is a single-line script – see my post above:

$ cat /tmp/test.py 
from git import Blob

[tucked]

SSCCE

python3.7 -m venv venv
venv/bin/pip install GitPython==3.1.24 mypy==0.910
vim sscce.py

import git
git.Repo

venv/bin/python sscce.py  # runs fine
venv/bin/mypy --strict sscce.py

Expected

Success: no issues found in 1 source file

Observed

sscce.py:2: error: Module has no attribute "Repo" Found 1 error in 1 file (checked 1 source file)

Notes

I suspect this is the same problem as https://github.com/pallets/click/issues/1879 (related: https://github.com/python/mypy/issues/10198). This is actually called out in https://github.com/gitpython-developers/GitPython/issues/1095#issuecomment-891111515 too.

[dmtucker]

What feedback is this waiting on?

[Byron]

Thanks for the hint, probably unintended.

[rra]

I believe the root cause of this issue is that the top-level __init__.py file of the git module dynamically constructs __all__. mypy relies on __all__ to know which symbols are intended to be exported and will complain about importing symbols from modules that don't appear to intend to export them (depending on the mypy configuration) because this is normally a bug: depending on an implementation detail of another module that isn't part of its interface and therefore may change. It's a Python parser but not a Python execution framework, and I don't think it runs enough Python to know what __all__ is set to at runtime.

Unfortunately, if I'm right, the fix may be to explicitly list all of the symbols intended to be exported from git instead of dynamically constructing them, which will require some ongoing duplicate bookkeeping when new symbols are added.