django-stubs icon indicating copy to clipboard operation
django-stubs copied to clipboard

Published 20 hours ago • verified publisher icon typeddjango

Mypy crash on querysets

Open matejsp opened this issue 3 years ago • 3 comments

Bug report

Mypy crashes

What's wrong

class SomeEvent(models.Model):

user_id = models.IntegerField(null=False, unique=True)
exception_name = models.CharField(max_length=200, null=True, blank=True)
exception_type = models.CharField(max_length=200, null=True, blank=True)

And then in code:

    data = (
        models.SomeEvent.objects.values('exception_name').order_by().annotate(Count('exception_name'))
    )

It crashes with:

some_handler.py:283: error: INTERNAL ERROR -- Please try using mypy master on GitHub:
https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
Please report a bug at https://github.com/python/mypy/issues
version: 0.971
Traceback (most recent call last):
  File "/Users/myuser/.virtualenvs/python38env/bin/mypy", line 8, in <module>
    sys.exit(console_entry())
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/__main__.py", line 12, in console_entry
    main(None, sys.stdout, sys.stderr)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/main.py", line 96, in main
    res, messages, blockers = run_build(sources, options, fscache, t0, stdout, stderr)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/main.py", line 173, in run_build
    res = build.build(sources, options, None, flush_errors, fscache, stdout, stderr)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/build.py", line 154, in build
    result = _build(
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/build.py", line 230, in _build
    graph = dispatch(sources, manager, stdout)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/build.py", line 2729, in dispatch
    process_graph(graph, manager)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/build.py", line 3087, in process_graph
    process_stale_scc(graph, scc, manager)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/build.py", line 3185, in process_stale_scc
    graph[id].type_check_first_pass()
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/build.py", line 2180, in type_check_first_pass
    self.type_checker().check_first_pass()
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checker.py", line 329, in check_first_pass
    self.accept(d)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checker.py", line 435, in accept
    stmt.accept(self)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/nodes.py", line 1039, in accept
    return visitor.visit_class_def(self)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checker.py", line 1823, in visit_class_def
    self.accept(defn.defs)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checker.py", line 435, in accept
    stmt.accept(self)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/nodes.py", line 1110, in accept
    return visitor.visit_block(self)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checker.py", line 2200, in visit_block
    self.accept(s)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checker.py", line 435, in accept
    stmt.accept(self)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/nodes.py", line 758, in accept
    return visitor.visit_func_def(self)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checker.py", line 786, in visit_func_def
    self._visit_func_def(defn)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checker.py", line 790, in _visit_func_def
    self.check_func_item(defn, name=defn.name)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checker.py", line 852, in check_func_item
    self.check_func_def(defn, typ, name)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checker.py", line 1037, in check_func_def
    self.accept(item.body)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checker.py", line 435, in accept
    stmt.accept(self)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/nodes.py", line 1110, in accept
    return visitor.visit_block(self)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checker.py", line 2200, in visit_block
    self.accept(s)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checker.py", line 435, in accept
    stmt.accept(self)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/nodes.py", line 1176, in accept
    return visitor.visit_assignment_stmt(self)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checker.py", line 2242, in visit_assignment_stmt
    self.check_assignment(s.lvalues[-1], s.rvalue, s.type is None, s.new_syntax)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checker.py", line 2436, in check_assignment
    rvalue_type = self.expr_checker.accept(rvalue)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checkexpr.py", line 3999, in accept
    typ = node.accept(self)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/nodes.py", line 1771, in accept
    return visitor.visit_call_expr(self)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checkexpr.py", line 306, in visit_call_expr
    return self.visit_call_expr_inner(e, allow_none_return=allow_none_return)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checkexpr.py", line 355, in visit_call_expr_inner
    callee_type = get_proper_type(self.accept(e.callee, type_context, always_allow_any=True))
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checkexpr.py", line 3999, in accept
    typ = node.accept(self)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/nodes.py", line 1691, in accept
    return visitor.visit_member_expr(self)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checkexpr.py", line 2078, in visit_member_expr
    result = self.analyze_ordinary_member_access(e, is_lvalue)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checkexpr.py", line 2089, in analyze_ordinary_member_access
    original_type = self.accept(e.expr)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checkexpr.py", line 3999, in accept
    typ = node.accept(self)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/nodes.py", line 1771, in accept
    return visitor.visit_call_expr(self)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checkexpr.py", line 306, in visit_call_expr
    return self.visit_call_expr_inner(e, allow_none_return=allow_none_return)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checkexpr.py", line 355, in visit_call_expr_inner
    callee_type = get_proper_type(self.accept(e.callee, type_context, always_allow_any=True))
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checkexpr.py", line 3999, in accept
    typ = node.accept(self)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/nodes.py", line 1691, in accept
    return visitor.visit_member_expr(self)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checkexpr.py", line 2078, in visit_member_expr
    result = self.analyze_ordinary_member_access(e, is_lvalue)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checkexpr.py", line 2089, in analyze_ordinary_member_access
    original_type = self.accept(e.expr)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checkexpr.py", line 3999, in accept
    typ = node.accept(self)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/nodes.py", line 1771, in accept
    return visitor.visit_call_expr(self)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checkexpr.py", line 306, in visit_call_expr
    return self.visit_call_expr_inner(e, allow_none_return=allow_none_return)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checkexpr.py", line 384, in visit_call_expr_inner
    ret_type = self.check_call_expr_with_callee_type(callee_type, e, fullname,
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checkexpr.py", line 893, in check_call_expr_with_callee_type
    ret_type, callee_type = self.check_call(
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checkexpr.py", line 958, in check_call
    return self.check_callable_call(callee, args, arg_kinds, context, arg_names,
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checkexpr.py", line 1086, in check_callable_call
    new_ret_type = self.apply_function_plugin(
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checkexpr.py", line 773, in apply_function_plugin
    return method_callback(
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy_django_plugin/transformers/querysets.py", line 272, in extract_proper_type_queryset_values
    assert isinstance(default_return_type, Instance)

How is that should be

It should not crash

System information

  • OS: MacOSX 12.5.1
  • python version: 3.8.13
  • django version: 2.2.x
  • mypy version: 0.971
  • django-stubs version: 1.12.0
  • django-stubs-ext version: 0.5.0

matejsp avatar Sep 19 '22 08:09 matejsp

3.71 isn't a real Mypy version, did you mean 0.971 maybe?

adamchainz avatar Sep 19 '22 08:09 adamchainz

yes sry: mypy==0.971

matejsp avatar Sep 19 '22 09:09 matejsp

And another crash:

services.py:59: error: INTERNAL ERROR -- Please try using mypy master on GitHub:
https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
Please report a bug at https://github.com/python/mypy/issues
version: 0.971
Traceback (most recent call last):
  File "/Users/myuser/.virtualenvs/python38env/bin/mypy", line 8, in <module>
    sys.exit(console_entry())
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/__main__.py", line 12, in console_entry
    main(None, sys.stdout, sys.stderr)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/main.py", line 96, in main
    res, messages, blockers = run_build(sources, options, fscache, t0, stdout, stderr)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/main.py", line 173, in run_build
    res = build.build(sources, options, None, flush_errors, fscache, stdout, stderr)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/build.py", line 154, in build
    result = _build(
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/build.py", line 230, in _build
    graph = dispatch(sources, manager, stdout)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/build.py", line 2729, in dispatch
    process_graph(graph, manager)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/build.py", line 3087, in process_graph
    process_stale_scc(graph, scc, manager)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/build.py", line 3185, in process_stale_scc
    graph[id].type_check_first_pass()
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/build.py", line 2180, in type_check_first_pass
    self.type_checker().check_first_pass()
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checker.py", line 329, in check_first_pass
    self.accept(d)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checker.py", line 435, in accept
    stmt.accept(self)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/nodes.py", line 758, in accept
    return visitor.visit_func_def(self)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checker.py", line 786, in visit_func_def
    self._visit_func_def(defn)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checker.py", line 790, in _visit_func_def
    self.check_func_item(defn, name=defn.name)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checker.py", line 852, in check_func_item
    self.check_func_def(defn, typ, name)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checker.py", line 1037, in check_func_def
    self.accept(item.body)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checker.py", line 435, in accept
    stmt.accept(self)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/nodes.py", line 1110, in accept
    return visitor.visit_block(self)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checker.py", line 2200, in visit_block
    self.accept(s)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checker.py", line 435, in accept
    stmt.accept(self)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/nodes.py", line 1176, in accept
    return visitor.visit_assignment_stmt(self)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checker.py", line 2242, in visit_assignment_stmt
    self.check_assignment(s.lvalues[-1], s.rvalue, s.type is None, s.new_syntax)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checker.py", line 2436, in check_assignment
    rvalue_type = self.expr_checker.accept(rvalue)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checkexpr.py", line 3999, in accept
    typ = node.accept(self)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/nodes.py", line 1771, in accept
    return visitor.visit_call_expr(self)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checkexpr.py", line 306, in visit_call_expr
    return self.visit_call_expr_inner(e, allow_none_return=allow_none_return)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checkexpr.py", line 355, in visit_call_expr_inner
    callee_type = get_proper_type(self.accept(e.callee, type_context, always_allow_any=True))
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checkexpr.py", line 3999, in accept
    typ = node.accept(self)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/nodes.py", line 1691, in accept
    return visitor.visit_member_expr(self)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checkexpr.py", line 2078, in visit_member_expr
    result = self.analyze_ordinary_member_access(e, is_lvalue)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checkexpr.py", line 2089, in analyze_ordinary_member_access
    original_type = self.accept(e.expr)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checkexpr.py", line 3999, in accept
    typ = node.accept(self)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/nodes.py", line 1771, in accept
    return visitor.visit_call_expr(self)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checkexpr.py", line 306, in visit_call_expr
    return self.visit_call_expr_inner(e, allow_none_return=allow_none_return)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checkexpr.py", line 384, in visit_call_expr_inner
    ret_type = self.check_call_expr_with_callee_type(callee_type, e, fullname,
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checkexpr.py", line 893, in check_call_expr_with_callee_type
    ret_type, callee_type = self.check_call(
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checkexpr.py", line 958, in check_call
    return self.check_callable_call(callee, args, arg_kinds, context, arg_names,
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checkexpr.py", line 1086, in check_callable_call
    new_ret_type = self.apply_function_plugin(
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy/checkexpr.py", line 773, in apply_function_plugin
    return method_callback(
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy_django_plugin/transformers/querysets.py", line 231, in extract_proper_type_queryset_annotate
    annotated_type = get_or_create_annotated_type(api, model_type, fields_dict=fields_dict)
  File "/Users/myuser/.virtualenvs/python38env/lib/python3.8/site-packages/mypy_django_plugin/transformers/models.py", line 576, in get_or_create_annotated_type
    model_module_file = api.modules[model_module_name]  # type: ignore
KeyError: 'django_stubs_ext'
services.py:59: : note: use --pdb to drop into pdb

    result = models.Account.objects.annotate(
        is_xxxx=Case(
            When(type__in=XXX_TYPES, then=True),
            default=False,
            output_field=BooleanField(),
        )
    ).annotate(
        full_name=Case(
            When(
                Q(account__user__someboolfield=True)
                & Q(account__user__company__isnull=False),
                then='account__user__company',
            ),
            When(
                Q(account__user__first_name='') & Q(account__user__last_name=''),
                then='account__user__username',
            ),
            default=Concat('account__user__first_name', Value(' '), 'account__user__last_name'),
        )
    )

matejsp avatar Sep 19 '22 19:09 matejsp

same thing happened for me.

mypy output:

$ mypy --show-column-numbers --config-file ~/code/python/miare/mypy.ini --follow-imports=skip ~/code/python/miare/capacity_prediction/
[...]
/home/sazary/code/python/miare/capacity_prediction/services/utils/occasion_applier.py:23: error: INTERNAL ERROR -- Please try using mypy master on GitHub:
https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
Please report a bug at https://github.com/python/mypy/issues
version: 0.991
Traceback (most recent call last):
  File "mypy/checkexpr.py", line 4665, in accept
  File "mypy/nodes.py", line 1841, in accept
  File "mypy/checkexpr.py", line 410, in visit_call_expr
  File "mypy/checkexpr.py", line 530, in visit_call_expr_inner
  File "mypy/checkexpr.py", line 1181, in check_call_expr_with_callee_type
  File "mypy/checkexpr.py", line 1264, in check_call
  File "mypy/checkexpr.py", line 1466, in check_callable_call
  File "mypy/checkexpr.py", line 1021, in apply_function_plugin
  File "/home/sazary/.virtualenvs/miare39/lib/python3.9/site-packages/mypy_django_plugin/transformers/querysets.py", line 149, in extract_proper_type_queryset_values_list
    assert isinstance(default_return_type, Instance)
AssertionError:
/home/sazary/code/python/miare/capacity_prediction/services/utils/occasion_applier.py:23: : note: use --pdb to drop into pdb

the part of my code that mypy crashes on (occasion_applier.py lines 22 & 23):

        for occasion in occasions:
            occasion_area_ids = list(occasion.areas.values_list('id', flat=True))

I don't use any custom manager for Occasion model, nor for Area. The related part is:

class Occasion(models.Model):
    [...]
    areas = models.ManyToManyField(to=DriverArea, related_name='+', verbose_name='مناطق')
    [...]

My mypy.ini file:

[mypy]
python_version = 3.9
ignore_missing_imports = True
show_traceback = True
incremental = True

plugins =
    mypy_django_plugin.main,
    mypy_drf_plugin.main
    ; numpy.typing.mypy_plugin

[mypy.plugins.django-stubs]
django_settings_module = biar.settings.development

versions:

django-stubs==1.13.0
django-stubs-ext==0.7.0
djangorestframework-stubs==1.7.0
mypy==0.991
mypy-extensions==0.4.3
Django==3.2.2

Python 3.9.15
OS: Ubuntu 22.04.1

monkey patching is enabled too.

sazary avatar Nov 17 '22 13:11 sazary

Same error as #701 . I tried upgrading to master and i can still replicate it.

sidmitra avatar Dec 08 '22 22:12 sidmitra