Reimplement popular Mypy Plugins

[johnthagen]

For better discoverability of the decision, cross post to document reasons for

• https://github.com/astral-sh/ruff/issues/15828

Description

In order to migrate from Mypy to Ruff for static type checking, one thing users may need are reimplementations of the most popular Mypy plugins.

• https://mypy.readthedocs.io/en/stable/extending_mypy.html#extending-mypy-using-plugins

Assuming for a moment that exposing a Mypy-compatible plugin interface is out of scope, the following Mypy plugins are ones I've seen in wide use within the community and could be considered:

• Some appear built into Mypy already: https://github.com/python/mypy/tree/master/mypy/plugins
• (Deprecated) Numpy: https://numpy.org/devdocs/reference/typing.html#mypy-plugin
• Pydantic: https://docs.pydantic.dev/latest/integrations/mypy/#enabling-the-plugin
• django-stubs: https://github.com/typeddjango/django-stubs?tab=readme-ov-file#installation
• djangorestframework-stubs: https://github.com/typeddjango/djangorestframework-stubs?tab=readme-ov-file#installation

Another list of Mypy plugins I found:

• https://github.com/typeddjango/awesome-python-typing?tab=readme-ov-file#mypy-plugins

Related to

• https://github.com/astral-sh/ruff/issues/3893

References

• https://news.ycombinator.com/item?id=43919746
• https://www.youtube.com/live/XVwpL_cAvrw?feature=shared&t=2528

[max-muoto]

Implementation of a Pydantic plugin probably shouldn't be necessary post dataclass transforms - effectively every single behavior is natively supported in the typing system at this point: https://peps.python.org/pep-0681/

[max-muoto]

I think this also begs a more philosophical question around the direction of Ty. Some type-checkers such as Pyright have avoided implementing plugin support to instead push for extending the native type-system: https://github.com/microsoft/pyright/issues/637#issuecomment-632759717

This in turn led to the aforementioned dataclass transforms. I'm certainly sympathetic to lack of robust type-checking for Django for non-MyPy type-checkers but https://github.com/sbdchd/django-types does a good job of typing Django without the need for any custom extensions from my experience.

[carljm]

We also prefer a well-specified inter-operable type system over plugins. We currently have no plans to build a plugin system for ty. Implementing special-cased support for some widely-used library is more plausible, but also not on the current roadmap, and something we would prefer to avoid in favor of adding a general means to the type system to support the needed pattern(s).

[w0rp]

I will add in that based on my professional on the job experience with Django for 11 years I have viewed typing Django as pretty much a lost cause. Django dev teams typically aren't interested in adopting type checking. I had previously success re-tooling Django classes to add in type information that gave better results than the mypy Django plugins.

In my opinion: Django was built a long time ago long before PEP 484, and it will take significant efforts from the Django team to make breaking changes to Django to make it easier to produce accurate types for QuerySet classes, Manager classes, and that's even without a refactoring I think is needed for better asyncio support and avoiding the N+1 query problem.

I'd probably say: avoid adopting a plugin architecture for some time to come and focus on achieving feature parity with pyright.

[lexicalunit]

+1 to feature party with pyright and to not adding plugins. Getting the most basic things working with https://github.com/sbdchd/django-types (django-stubs but with the mypy plugin stripped out) would be ideal.

Right now I get issues with code as simple as:

from django.db import models

class Foo(models.Model):
    id: int  # even with explicitly added annotations it still doesn't work
    name = models.TextField()

def bar():
    f = Foo.objects.create()
    print(f.id)
    print(f.name)

error[unresolved-attribute]: Type `Self` has no attribute `id`
  --> my/test/file:9:11
   |
25 | def bar():
26 |     f = Foo.objects.create()
27 |     print(f.id)
   |           ^^^^
28 |     print(f.name)
   |
info: rule `unresolved-attribute` is enabled by default

error[unresolved-attribute]: Type `Self` has no attribute `name`
  --> my/test/file:10:11
   |
26 |     f = Foo.objects.create()
27 |     print(f.id)
28 |     print(f.name)
   |           ^^^^^^
   |
info: rule `unresolved-attribute` is enabled by default

[jellis18]

So it appears that django-types is incompatible with ty at the moment? Has anyone found a way to get it to work?

[w0rp]

So it appears that django-types is incompatible with ty at the moment? Has anyone found a way to get it to work?

See my comment above. In my experience re-implementing classes in Django with explicit declarations and # type: ignore re-declarations of fields in Model classes works better. In combination with django-types it's the best you will ever do. The same is true for Pyright, which backs Pylance.

Django is sorely in need of a new major version with first-class type support via a dataclass re-implementation of Django models and async def methods for QuerySet classes with a removal of the magic addition of QuerySet methods to Manager classes. (Would it really kill people to write Foo.objects.all().bar() instead of Foo.objects.bar()?)

I use FastAPI for all new APIs instead of Django and DRF. I recognise the ongoing need to maintain Django code, plus Django's ease-of-use for implementing interactive blog and news sites, forums, admin panels, etc.

[matmair]

No Django support will hold this back in a mayor part of the python ecosystem. You need to disable some pretty basic rules to use ty with Django right now. This is one of the few areas where Mypy still has the edge.

[AdityaMayukhSom]

Implementation of a Pydantic plugin probably shouldn't be necessary post dataclass transforms - effectively every single behavior is natively supported in the typing system at this point: https://peps.python.org/pep-0681/

Any thoughts on how to enable it to understand the kwargs for initialization on BaseModel implementations?

Keeps on throwing Argument description does not match any known parameterty[unknown-argument](https://ty.dev/rules#unknown-argument)

[carljm]

@AdityaMayukhSom Might be https://github.com/astral-sh/ty/issues/1425 -- are those fields on Group not type-annotated, by any chance?

If it's not that, you'd probably need to open a dedicated issue and show more of your code for a full repro.

[AdityaMayukhSom]

@AdityaMayukhSom Might be #1425 -- are those fields on Group not type-annotated, by any chance?

If it's not that, you'd probably need to open a dedicated issue and show more of your code for a full repro.

All are type annotated. @carljm P.S.: TypeId is alias for uuid.UUID, defined as TypeId = uuid.UUID

[carljm]

@AdityaMayukhSom Can you create a new issue (not a comment on this issue)? Thank you! If you are able to minimize the problem to the smallest example where it still reproduces (and show full code for that example, including all base classes) that would be super helpful.