Feature request: add mypy static type checking support

[bpkroth]

error: Skipping analyzing "json5": module is installed, but missing library stubs or py.typed marker

[dpranke]

Hi! Sorry for the delay in responding :( This seems like a good idea and I will try to get to this as soon as I can.

[richardxia]

I'd be interested in helping out with this. A question I have is whether you intend to continue supporting Python 2, as currently described in the package metadata. Being able to use the type annotation syntax in Python 3 makes it much easier to maintain.

[dpranke]

I don't intend to support Python 2 any more; I need to update the package metadata.

If someone wanted to work on this and send me a pull request, that would be fine. I basically want to do whatever the json standard library does. As far as I can tell, there is no official typing yet. The issue is discussed in https://github.com/python/typing/issues/182 .

There is a complication, as raised in https://github.com/python/typing/issues/182#issuecomment-199910414 : because of the object_hook and other custom decoders, it's not clear to me that you can correctly do anything other than Any for the value type(s). So, I'm not sure what to do here (which is why I haven't done anything yet).

[richardxia]

I don't intend to support Python 2 any more; I need to update the package metadata.

That's good to hear.

If someone wanted to work on this and send me a pull request, that would be fine. I basically want to do whatever the json standard library does. As far as I can tell, there is no official typing yet. The issue is discussed in python/typing#182 .

Yes, that's correct. I believe that the Python core maintainers are hesitant to introduce type annotations to the official libraries because of concerns that it may 1) slow down all Python users, regardless of whether they use static type checkers or not, and 2) it could cause users who are using static type checkers to start hitting (genuine) type errors with no way of rolling back because it would be tied to the official CPython interpreter release. https://discuss.python.org/t/type-annotations-in-the-standard-library/7309 is the closest discussion that I could find online.

I believe, however, that the typeshed repo is semi-blessed, and it avoids the issues above by being something that someone would have to manually install and it has a release cadence that's decoupled from the CPython interpreter. Notably, mypy vendors in typeshed, and it appears that they regularly sync it with the upstream repo. Typeshed has type stubs for most of the standard library, including the json package: https://github.com/python/typeshed/tree/2f65ca5cd4feaa1c69953ef7681416d8d435ffa7/stdlib/json

Re: object_hook and other similar things, the typeshed stubs for json just use Any in many places, which I think is reasonable. It's still useful to have type stubs even with the Any type because you can still check:

• that a function exists at all (e.g. that the pyjson5 module has dump, dumps, load, loads functions)
• that a function has a specific set of arguments (e.g. preventing people from passing in keyword arguments that don't exist)
• that you still get type safety on a subset of the arguments (e.g. the fp argument is a file-like object)

---

If this seems reasonable to you, then I think I have one remaining question for you: Would you prefer to use inline annotations, using the : syntax available in Python 3.5+, or would you prefer to use type stubs (.pyi files)? I personally prefer inline annotations, since it's less difficult for it to get out of sync with the codebase, but if you wanted to more closely match how typeshed does it for the standard library json package, then you could use type stubs.

[dpranke]

Sheesh. I can't believe it's been almost 6 weeks and I haven't replied. I'm very sorry!

I don't have a strong opinion (yet?) about how to go about things. I suppose what I would like to see would be a proposal for what the typed interface would actually look like. I also need to understand more about how mypy works (particularly w/ typeshed).

The thing I'm least certain about is how any implementation we do might lock us in such that we can't change or improve it in the future, due to compatibility concerns.

[dpranke]

Is there some way you can tell mypy to ignore or override particular stubs for a package in the typeshed?

[dpranke]

@richardxia - Are you still out there and interested in this? I've created #71 for what I think this should look like but I'm a complete newbie at python typing and could use some feedback.

[richardxia]

Is there some way you can tell mypy to ignore or override particular stubs for a package in the typeshed?

Sorry for missing this question in the past, but there is a way to do this, but I think I need to clarify a few things first.

Typeshed is a special GitHub repository that holds type annotations for both the Python standard library and some third party libraries, though the latter is mostly community-driven. If you add type annotations directly to your package's source code, then your type annotations don't live in typeshed. You can view typeshed as being a last resort place for type annotations to exist.

PEP 561 outlines a specific order of precedence when it comes to loading type annotations, where the general rule is that you want to give the client user the final say over things. In this ordering, typeshed is actually last, and type annotations that exist directly in the source code of a package will be preferred over typeshed. These rules are supposed to be followed by all type checkers, not mypy, so to override type annotations, you usually want to follow this approach first.

That said, mypy does provide options for overriding type annotations. There is a MYPYPATH environment variable/mypy_pyath config file option for setting the path for where mypy should discover type annotation files, similar to how PYTHONPATH works for Python imports. Also, mypy provides the ability to set its configuration options on a per package basis, via its overrides system, so if you needed to disable type checking for a specific package, you can do so.

@richardxia - Are you still out there and interested in this? I've created #71 for what I think this should look like but I'm a complete newbie at python typing and could use some feedback.

Sorry again for the radio silence on this, but thanks again for submitting that PR! I left some review comments on it, but I'm happy to continue providing feedback and answering questions there.

[dpranke]

I think I'm all set with the latest set of patches that should address all of your feedback. If you felt like taking another look at the PR, that'd be great, but otherwise I'll plan to merge it in in a couple days if I don't hear from you.

[dpranke]

implemented and released as v0.9.18.