Fixed type errors in `mypy` GitHub Action

[lukeconibear]

trafficstars

Fixed type errors in mypy GitHub Action

• [x] Closes #6962

Running the GitHub Action for mypy locally returns successful:

$ python -m mypy --install-types --non-interactive 
Success: no issues found in 144 source files

[headtr1ck]

That is weird, with from __future__ import annotations this should work.

Actually the pyupgrade pre-commit hook should even replace the typing versions by built-in versions.

[headtr1ck]

Could you add an additional mypy workflow to test this behavior? See https://github.com/pydata/xarray/issues/6962#issuecomment-1277287183

[headtr1ck]

When I run locally mypy --python-version 3.8 i get more errors. Maybe merge current main and let's see what new issues emerge.

[headtr1ck]

@dcherian on a different issue: can we prevent that the auto labeler removes manually added labels?

[dcherian]

can we prevent that the auto labeler removes manually added labels?

no idea. cc @TomNicholas

[headtr1ck]

Seems that pyupgrade does not update tuple -> Tuple inside of Union, which is exactly what caused these problems.

Looks good to me now 👍

[headtr1ck]

Seems like dasks typing is also not compatible with python 3.8, haha

[headtr1ck]

I cannot reproduce the mypy error locally. Does this break for someone else?

[max-sixty]

I get some errors:

❯ git describe
v0.17.0-798-g3599f873

❯ mypy --python-version 3.8
xarray/core/types.py:72: error: "tuple" is not subscriptable  [misc]
xarray/core/types.py:74: error: "tuple" is not subscriptable  [misc]
xarray/core/types.py:76: error: "tuple" is not subscriptable  [misc]
xarray/core/types.py:78: error: "list" is not subscriptable  [misc]
xarray/core/_reductions.py:18: error: Cannot find implementation or library stub for module named "flox"  [import]
xarray/core/_reductions.py:18: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
xarray/core/merge.py:43: error: "tuple" is not subscriptable  [misc]
xarray/core/merge.py:44: error: "tuple" is not subscriptable  [misc]
xarray/core/merge.py:45: error: "tuple" is not subscriptable  [misc]
xarray/core/groupby.py:661: error: Cannot find implementation or library stub for module named "flox.xarray"  [import]
xarray/backends/api.py:65: error: "dict" is not subscriptable  [misc]
Found 10 errors in 5 files (checked 139 source files)

(not sure why git describe is showing that tag but the commit hash should repro)

[headtr1ck]

Hmmm, these errors should be fixed in the latest version of this PR.

The problem CI is running into is that mypy runs into an unrecoverable error parsing dask, this I cannot reproduce locally.

[max-sixty]

Hmmm, these errors should be fixed in the latest version of this PR.

Sorry, my test was on main. I can rerun on this PR.

[max-sixty]

I can't repro the failure on the current PR v0.17.0-804-g7018b950. I'm using Python 3.9 with the latest dask.

/home/runner/micromamba/envs/xarray-tests/lib/python3.10/site-packages/dask/utils.py:366: error: Parenthesized context managers are only supported in Python 3.9 and greater  [syntax]
Found 1 error in 1 file (errors prevented further checking)

I'm not sure why it's raising an error for an imported module where --follow-imports=silent. Isn't that supposed to suppress errors in imports?

And weirdly, I get errors running mypy in dask latest main, but not this error!

❯ mypy --python-version 3.8 .
dask/system.py:8: error: Unused "type: ignore" comment
dask/widgets/__init__.py:20: error: All conditional function variants must have identical signatures  [misc]
dask/widgets/__init__.py:23: error: All conditional function variants must have identical signatures  [misc]
dask/layers.py:1322: error: Unused "type: ignore" comment
dask/layers.py:1323: error: Unused "type: ignore" comment
dask/array/core.py:5814: error: Argument 1 to "ndindex" has incompatible type "Tuple[int, ...]"; expected "SupportsIndex"  [arg-type]
dask/array/fft.py:196: error: Unused "type: ignore" comment
dask/dataframe/core.py:438: error: Unused "type: ignore" comment
dask/dataframe/core.py:4186: error: Unused "type: ignore" comment
dask/dataframe/core.py:4197: error: Unused "type: ignore" comment
dask/dataframe/core.py:4209: error: Unused "type: ignore" comment
dask/dataframe/core.py:4408: error: Unused "type: ignore" comment
dask/dataframe/core.py:4420: error: Unused "type: ignore" comment
dask/dataframe/core.py:4425: error: Unused "type: ignore" comment
dask/dataframe/groupby.py:1204: error: Unused "type: ignore" comment
dask/dataframe/io/csv.py:9: error: Unused "type: ignore" comment
dask/bytes/tests/test_s3.py:380: error: Unused "type: ignore" comment
dask/bytes/tests/test_local.py:191: error: Unused "type: ignore" comment
dask/bag/tests/test_text.py:33: error: Unused "type: ignore" comment
dask/dataframe/tests/test_shuffle.py:1408: error: Unused "type: ignore" comment
dask/dataframe/tests/test_shuffle.py:1409: error: Unused "type: ignore" comment
dask/dataframe/tests/test_shuffle.py:1412: error: Unused "type: ignore" comment
dask/diagnostics/tests/test_profiler.py:21: error: Unused "type: ignore" comment
dask/tests/test_graph_manipulation.py:106: error: "tuple" is not subscriptable, use "typing.Tuple" instead  [misc]
Found 24 errors in 14 files (checked 243 source files)

[headtr1ck]

It seems that mypy encountered an unrecoverable runtime error. We should probably report that to mypy, but I cannot reproduce it locally, yet alone create a MVCE.

[Illviljan]

Dask is on the ignore list: https://github.com/pydata/xarray/blob/d6671dd414370d006254ba3156cb96256ce0e9c7/pyproject.toml#L31-L43

This seems to ignore the list and follow-imports doesn't seem to work either:

      - name: Run mypy with python3.8
        # silent all imports, since external repos might not support this
        run: |
          python -m mypy --install-types --non-interactive --python-version 3.8 --follow-imports=silent

A good ol' copy/paste job works as expected though. :) I think we can discuss more elegant solutions in a follow up PR.

[headtr1ck]

A good ol' copy/paste job works as expected though. :) I think we can discuss more elegant solutions in a follow up PR.

So now mypy is not crashing anymore? Thats weird, we should open an issue on mypy about this...

[Illviljan]

A good ol' copy/paste job works as expected though. :) I think we can discuss more elegant solutions in a follow up PR.

So now mypy is not crashing anymore? Thats weird, we should open an issue on mypy about this...

This one throws errors still: python -m mypy --install-types --non-interactive --python-version 3.8 --follow-imports=silent Did it ever crash if we used the normal mypy CI but with changed python version?

[headtr1ck]

This one throws errors still: python -m mypy --install-types --non-interactive --python-version 3.8 --follow-imports=silent Did it ever crash if we used the normal mypy CI but with changed python version?

I don't think we ever used --python-version outside of this PR.

We should report this mypy bug...

[Illviljan]

Thanks, @lukeconibear !