Cannot mix URL patterns and included urlconfs

[sephii]

Bug report

What's wrong

I cannot find a way to get the typing in urls.py right. It seems that a path that points to a view returns a URLPattern but a path that points to an included urlconf returns a URLResolver. So mixing both in a single list results in a List[object], for which mypy complains if you try to concatenate it with another list of url patterns.

For example the following:

urlpatterns = [
    path("admin/", admin.site.urls),
]

if settings.DEBUG:
    urlpatterns = (
        [
            path(
                "media/<path:path>/",
                django.views.static.serve,
                {"document_root": settings.MEDIA_ROOT, "show_indexes": True},
            ),
            path("__debug__/", include(debug_toolbar.urls)),
        ]
        + staticfiles_urlpatterns()
        + urlpatterns
    )

Results in:

myproject/config/urls.py: error: Unsupported operand types for + ("List[object]" and "List[URLPattern]")
myproject/config/urls.py: error: Incompatible types in assignment (expression has type "List[object]", variable has type "List[URLResolver]")
myproject/config/urls.py: error: Unsupported operand types for + ("List[object]" and "List[URLResolver]")

How is that should be

The code above shouldn’t trigger any errors, or maybe a section could be added to the FAQ explaining how to get typing right in urlconfs.

System information

• OS: NixOS 20.09
• python version: 3.7.9
• django version: 3.1
• mypy version: 0.790
• django-stubs version: 1.7.0

[sobolevn]

Yes, this is probably because List is invariant.

PR with a fix is welcome!

[micheller]

I used this snippet, to bypass the problem:

from django.urls import URLResolver, URLPattern


URL = typing.Union[URLPattern, URLResolver]
URLList = typing.List[URL]

#<...>
urlpatterns: URLList = [
    path('', include(router.urls)),
]

[sjdemartini]

The workaround from micheller worked for me for django-stubs 1.8.0, but it appears that this is no longer necessary in the newly-released django-stubs 1.9.0.

[sobolevn]

Awesome! Thanks for the info

[aleehedl]

Was it fixed though? I still experience the same error with django-stubs 1.9.0 installed. Also, I couldn't find any related commit that would fix it.

[sjdemartini]

Hm, it certainly no longer shows any errors for me after upgrading to django-stubs 1.9.0 and mypy 0.910 (from 0.812); I've removed all manual typing from my urls.py.

[sobolevn]

@aleehedl can you please share a minimal sample to reproduce this?

[danihodovic]

Here you go:

from django.conf import settings
from django.conf.urls.static import static
from django.contrib import admin
from django.urls import include, path
from django.views import defaults as default_views

urlpatterns = [
    path("health/", include("health_check.urls")),
    path(settings.ADMIN_URL, admin.site.urls),
    path("captcha/", include("captcha.urls")),
    path("", include("hodovi_ch.web.urls")),
    # Your stuff: custom urls includes go here
] + static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT)

if settings.DEBUG:
    # This allows the error pages to be debugged during development, just visit
    # these url in browser to see how these error pages look like.
    urlpatterns += [
        path(
            "400/",
            default_views.bad_request,
            kwargs={"exception": Exception("Bad Request!")},
        ),
        path(
            "403/",
            default_views.permission_denied,
            kwargs={"exception": Exception("Permission Denied")},
        ),
        path(
            "404/",
            default_views.page_not_found,
            kwargs={"exception": Exception("Page not Found")},
        ),
        path("500/", default_views.server_error),
    ]
    if "debug_toolbar" in settings.INSTALLED_APPS:
        import debug_toolbar

        urlpatterns = [path("__debug__/", include(debug_toolbar.urls))] + urlpatterns

$ poetry show mypy
name         : mypy
version      : 0.910
description  : Optional static typing for Python

dependencies
 - mypy-extensions >=0.4.3,<0.5.0
 - toml *
 - typed-ast >=1.4.0,<1.5.0
 - typing-extensions >=3.7.4

$ poetry show django-stubs
name         : django-stubs
version      : 1.9.0
description  : Mypy stubs for Django

dependencies
 - django *
 - django-stubs-ext >=0.3.0
 - mypy >=0.910
 - toml *
 - types-pytz *
 - types-PyYAML *
 - typing-extensions *

[aleehedl]

A minimal urls.py which gives an error:

# Installed deps

$ pip freeze
asgiref==3.4.1
Django==3.2.7
django-stubs==1.9.0
django-stubs-ext==0.3.1
mypy==0.910
mypy-extensions==0.4.3
pytz==2021.1
sqlparse==0.4.2
toml==0.10.2
types-pytz==2021.1.2
types-PyYAML==5.4.10
typing-extensions==3.10.0.2


# urls.py
from django.conf import settings
from django.conf.urls.static import static
from django.contrib import admin
from django.urls import path

urlpatterns = [
    path(settings.ADMIN_URL, admin.site.urls),
] + static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT)


# Then run mypy
$ mypy <project>
> urls.py:8: error: Unsupported operand types for + ("List[URLResolver]" and "List[URLPattern]")

[sobolevn]

Thanks! Looks like the problem is in + static(). Any PRs that can fix it are welcome.

[graingert]

I use * unpacking

from django.conf import settings
from django.conf.urls.static import static
from django.contrib import admin
from django.urls import include, path, URLPattern, URLResolver
from django.views import defaults as default_views

urlpatterns: list[URLPattern, URLResolver] = [
    path("health/", include("health_check.urls")),
    path(settings.ADMIN_URL, admin.site.urls),
    path("captcha/", include("captcha.urls")),
    path("", include("hodovi_ch.web.urls")),
    # Your stuff: custom urls includes go here
    *static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT)
]

if settings.DEBUG:
    # This allows the error pages to be debugged during development, just visit
    # these url in browser to see how these error pages look like.
    urlpatterns = [
        *urlpatterns,
        path(
            "400/",
            default_views.bad_request,
            kwargs={"exception": Exception("Bad Request!")},
        ),
        path(
            "403/",
            default_views.permission_denied,
            kwargs={"exception": Exception("Permission Denied")},
        ),
        path(
            "404/",
            default_views.page_not_found,
            kwargs={"exception": Exception("Page not Found")},
        ),
        path("500/", default_views.server_error),
    ]
    if "debug_toolbar" in settings.INSTALLED_APPS:
        import debug_toolbar

        urlpatterns = [path("__debug__/", include(debug_toolbar.urls)), *urlpatterns]

[MrkGrgsn]

I am getting this same problem (django-stubs==1.120, mypy==0.971) if I do nested includes like this:

membership_urls = [
    re_path("overview/", MemberView.as_view(), ...),
]
project_urls =[
    re_path("overview/", ProjectView.as_view(), ...),
    path("membership/", include(membership_urls),
]
reveal_type(project_urls)  # List[object]
urlpatterns = [
    path("project/", include(project_urls)
]

There is a warning on the final include call:

Argument 1 to "include" has incompatible type "List[object]"; expected "Union[Union[str, Module, Sequence[Union[URLPattern, URLResolver]]], Tuple[Union[str, Module, Sequence[Union[URLPattern, URLResolver]]], str]]"

I understand it's mypy behaviour to infer List[object] for mixed-type lists so I'm not sure much can be done about it. Perhaps extension magic?

[graingert]

I think you need to explicitly include the type for project_urls because of the way Variance and Union interact with inference in mypy:

project_urls: Sequence[Union[URLPattern, URLResolver]]] = [...

[toniengelhardt]

I have this pattern:

api_urls = ([

    # Authentication:
    path('auth/signup/', csrf_exempt(SignupView.as_view()), name='signup'),
    path('auth/mlt/', MagicLinkView.as_view(), name='magic_link'),
    path('auth/activate/', ActivateAccountView.as_view(), name='activate'),
    path('auth/token/', TokenObtainPairView.as_view(), name='token_obtain_pair'),
    path('auth/token/refresh/', TokenRefreshView.as_view(), name='token_refresh'),
    path('auth/logout/', LogoutView.as_view(), name='logout'),
    path('auth/social/google/', GoogleLogin.as_view(), name='google_login'),
    path('auth/', include('dj_rest_auth.urls')),

    # Apps:
    path('organizations/', include('apps.organizations.urls')),
    path('outputs/', include('apps.outputs.urls')),
    path('payments/', include('apps.payments.urls')),
    path('platforms/', include('apps.platforms.urls')),
    path('projects/', include('apps.projects.urls')),
    path('prompts/', include('apps.prompts.urls')),
    path('users/', include('apps.users.urls')),
    path('variables/', include('apps.variables.urls')),

], 'api')

urlpatterns = [

    # Admin:
    path('admin/', admin.site.urls),

    # Auth:
    path('accounts/', include('allauth.urls')),

    # API:
    path('api-auth/', include('rest_framework.urls')),
    path('api/', include(api_urls)),

    *static(settings.MEDIA_URL_PUBLIC, document_root=settings.MEDIA_ROOT),
]

and am getting errors for all paths that use actual views with .as_view() and not includes:

Error:

[danielsamuels]

I'm still seeing this problem with these versions:

mypy==1.3.0
django-stubs==4.2.1
django-stubs-ext==4.2.1

Fix, as above, was to type the variable like so:

from typing import Sequence
from django.urls import URLPattern, URLResolver

apipatterns: Sequence[URLPattern | URLResolver] = []