asgiref
asgiref copied to clipboard
django
Typing for sync/async decorators
This is a first pass draft, fairly sure it's broken for python versions lower than 3.8.
Seems to work but i'm not sure how to test it right this moment, i'm used to using pyright and will have to investigate how to use mypy
One thing that confused me a bit is that the async_to_sync function accepts awaitable but it actually seems to be looking for a callable that returns an awaitable? I could use some clarification there.
I'm also not sure what naming convention to use for the TypeVar/Paramspec pairs.
Looks like I also introduce some formatting errors to correct!
No worries - take the time you need to get everything working! I might suggest you mark the PR as a draft, though, so that way you can move it to "ready for review" when you think you've got it all ready to go.
mypy seems not to support paramspec yet so this might take a little while unless you want to switch to pyright or pyre https://github.com/python/mypy/issues/8645
I'm not really feeling like switching typechecker right at this moment, unfortunately. Sorry for delaying this yet further!
This is a first pass draft, fairly sure it's broken for python versions lower than 3.8.
typing_extensions.ParamSpec has been backported to python 3.6 so it should still work fine.
Seems to work but i'm not sure how to test it right this moment, i'm used to using pyright and will have to investigate how to use mypy
mypy on pypi currently doesn't support ParamSpec, but now that https://github.com/python/mypy/issues/10883 and https://github.com/python/mypy/issues/10866 and https://github.com/python/mypy/issues/10862 have been merged the version on m*ster at least treats ParamSpec as Any so as soon as mypy releases this change will be no worse for mypy users.
One thing that confused me a bit is that the async_to_sync function accepts
awaitablebut it actually seems to be looking for a callable that returns an awaitable? I could use some clarification there.
That should definitely be a Callable[P, Awaitable[R]]
I'm also not sure what naming convention to use for the TypeVar/Paramspec pairs.
you can just use:
P = ParamSpec("P")
R = TypeVar("R")
you don't need define new ParamSpec placeholder variables for every use - you can use the same one over and over.
Looks like I also introduce some formatting errors to correct!
Made a little more progress.
- Switching to using the main mypy branch for now (to be resolved before merging).
- Mypy seems to not like combining paramspec with generics
- Future is only generic in 3.9+ and will throw exceptions in previous versions, removed it for now
Not entirely sure whats going on but the checks appears to have been running for many hours 😱
Future is only generic in 3.9+ and will throw exceptions in previous versions, removed it for now
You'll need to use a forward reference with "Future[R]"
3.9 Tests are passing and these are the mypy issues left. Most of which seem to be caused by mypy not recognizing Generic's can accept ParamSpecs.
asgiref/sync.py:126: error: Free type variable expected in Generic[...]
asgiref/sync.py:159: error: "Callable[..., Awaitable[R]]" has no attribute "__self__"
asgiref/sync.py:183: error: Name "P.args" is not defined
asgiref/sync.py:183: error: Name "P.kwargs" is not defined
asgiref/sync.py:310: error: Name "P.args" is not defined
asgiref/sync.py:311: error: Name "P.kwargs" is not defined
asgiref/sync.py:348: error: Free type variable expected in Generic[...]
asgiref/sync.py:432: error: Name "P.args" is not defined
asgiref/sync.py:432: error: Name "P.kwargs" is not defined
asgiref/sync.py:441: error: Argument 1 to "get" of "ContextVar" has incompatible type "None"; expected "Union[ContextVar[str], str]"
asgiref/sync.py:498: error: Returning Any from function declared to return "R"
asgiref/sync.py:506: error: Function is missing a return type annotation
asgiref/sync.py:512: error: Name "P.args" is not defined
asgiref/sync.py:513: error: Name "P.kwargs" is not defined
asgiref/sync.py:569: error: "SyncToAsync" expects 1 type argument, but 2 given
asgiref/sync.py:569: error: Invalid location for ParamSpec "P"
asgiref/sync.py:569: note: You can use ParamSpec as the first argument to Callable, e.g., 'Callable[P, int]'
asgiref/sync.py:578: error: "SyncToAsync" expects 1 type argument, but 2 given
asgiref/sync.py:578: error: Invalid location for ParamSpec "P"
asgiref/sync.py:578: note: You can use ParamSpec as the first argument to Callable, e.g., 'Callable[P, int]'
3.9 Tests are passing and these are the mypy issues left. Most of which seem to be caused by mypy not recognizing Generic's can accept ParamSpecs.
asgiref/sync.py:126: error: Free type variable expected in Generic[...] asgiref/sync.py:159: error: "Callable[..., Awaitable[R]]" has no attribute "__self__" asgiref/sync.py:183: error: Name "P.args" is not defined asgiref/sync.py:183: error: Name "P.kwargs" is not defined asgiref/sync.py:310: error: Name "P.args" is not defined asgiref/sync.py:311: error: Name "P.kwargs" is not defined asgiref/sync.py:348: error: Free type variable expected in Generic[...] asgiref/sync.py:432: error: Name "P.args" is not defined asgiref/sync.py:432: error: Name "P.kwargs" is not defined asgiref/sync.py:441: error: Argument 1 to "get" of "ContextVar" has incompatible type "None"; expected "Union[ContextVar[str], str]" asgiref/sync.py:498: error: Returning Any from function declared to return "R" asgiref/sync.py:506: error: Function is missing a return type annotation asgiref/sync.py:512: error: Name "P.args" is not defined asgiref/sync.py:513: error: Name "P.kwargs" is not defined asgiref/sync.py:569: error: "SyncToAsync" expects 1 type argument, but 2 given asgiref/sync.py:569: error: Invalid location for ParamSpec "P" asgiref/sync.py:569: note: You can use ParamSpec as the first argument to Callable, e.g., 'Callable[P, int]' asgiref/sync.py:578: error: "SyncToAsync" expects 1 type argument, but 2 given asgiref/sync.py:578: error: Invalid location for ParamSpec "P" asgiref/sync.py:578: note: You can use ParamSpec as the first argument to Callable, e.g., 'Callable[P, int]'
Is there a ticket for this in mypy? I think it's supposed to be ignored
Is there a ticket for this in mypy? I think it's supposed to be ignored
https://github.com/python/mypy/issues/11362 created one with minimal reproduction code
3.6 tests are failing with a circular import caused by the context shenanigans on Local. Looks kinda tricky to solve
@andrewgodwin @graingert I've updated this with a workaround based on the current (unreleased) mypy version. Temporarily installing mypy from the git version. Once mypy cuts a release this should be good to go. No generics required, but maybe a little hacky.
Other option is to wait for generic support in mypy and a release. Which who knows maybe that'll happen before the next release rendering the workaround redundant.
Will current versions of mypy-in-the-wild choke on the new stuff, btw, or will they ignore it and not typecheck it? Considering the type annotations here are mostly for the benefit of users rather than asgiref itself, I want to ensure it's not going to torpedo current users.
Sadly mypy 0.910 will choke on this. I think the work-around is to provide these aliases in a new module eg asgiref.sync_paramspec
Yes this will blow up on wild mypy. Once they roll out a new release this could be merged without too much worry unless you intend for backwards compatibility with older mypy versions.
Ugh, yeah, this is tricky because end-users mypy installations is the thing we care about at the end of the day. No way to somehow erase ParamSpec and fall back to Any or something on those older versions, I'm guessing?
mypy 0.930 is out with experimental support for ParamSpec! https://mypy-lang.blogspot.com/2021/12/mypy-0930-released.html?m=1
This is particularly good as it has support for Generic[P, R]
Given that Python 3.6 is no longer supported as of asgiref version 3.5.0, is now a good time to revisit this discussion? Sorry, I've just been following along here and am really excited to see this feature land at some point 😄
Maybe? I don't keep tight track of typing stuff right now so I'm not quite sure where things sit in terms of this working with 3.7.
All versions of python support ParamSpec, even Python 2.7 all you need to do is import it in an "if TYPE_CHECKING:" block, or use the typing_extensions backport