python
[Feature Request] TypeVar variance definition alias
Defining the two variances of a TypeVar is rather verbose:
T_co = TypeVar("T_co", covariant=True)
T_contra = TypeVar("T_contra", contravariant=True)
So I propose an alternative syntax, similar to that of Scala's variances:
T_co = +TypeVar("T_co")
T_contra = -TypeVar("T_contra")
Co- and contra-variance describe increasing and decreasing specificities of the typevars, respectively. So using + and - to symbolize these "directions" makes intuitive sense.
This was proposed before. At the time I’ve didn’t care much for it, but now I think it’s actually a good idea.
You probably shouldn’t use this on the TypeVar call though but rather in the use , e.g.
def f(a: +T): ...
A tricky thing will be introducing it, since TypeVar is imported from typing in the stdlib.
Yes, I like that better as well @gvanrossum .
From what I can see in PEP 484, variance only applies to generic types, so I suppose it'll be used as e.g.
class Spam(Generic[+T]): ...
If I understand correctly, introducing this would require it to go through the PEP trajectory, right? I wouldn't mind writing up a draft PEP if you think that it has a chance of being accepted :)
I just realized that there is an incompatibility, which could make the implementation tricky, but not impossible:
In the current api, the variance is a property of the TypeVar itself and has a unique name, e.g.
T_co = TypeVar("T_co", covariant=True)
class Spam(Generic[T_co]):
eggs: T_co
But when using the corresponding + syntax, T and +T are both named "T". Furthermore, the (in my opinion desirable) way that generic variances work in Scala, is that the + or - prefixes are used only in the generic type definition, e.g.
T = TypeVar("T")
class Spam(Generic[+T]):
eggs: T
From what I understand from the current typing.Generic and typing.TypeVar implementation, the variance of a generic class is implicitly determined from its TypeVars.
So a "naïve" implementation where +T evaluates to e.g. TypeVar("T", covariant=True), will cause the typechecker to see T and +T as being distinctly different, and will probably result in the misinterpretation of a duplicate declaration.
If this is in fact the case, it will require changing the implementation so that the generic class explicitly stores the variance of each of its TypeVars. Specifying the (co/contra)variance of a TypeVar with +/- should be only be done within the generic type arguments, e.g. class Spam(Generic[+T]).
I see that the implementation will be tricky. But this "incompatibility" can also be seen as a strong argument in favor of this feature:
In PEP 484 it says:
Note: Covariance or contravariance is not a property of a type variable, but a property of a generic class defined using this variable. Variance is only applicable to generic types; generic functions do not have this property. The latter should be defined using only type variables without covariant or contravariant keyword arguments.
Hence it's counterintuitive to have to mark the TypeVar itself as being co(ntra)variant.
I don't think that's a major concern, although it would require a bit of new coding in type checkers.
Most type checkers are static; they never evaluate the code. So they don't care what +T does at runtime: they just need their parsers to understand this syntax. For those that do evaluate the code at runtime, you're right that we need some way to trace +T back to T. I'd suggest implementing it by making +T return something like TypeVarWrapper(T, covariant=True).
I'm new to contributing here, but I'd like to give this one a shot. Is my assumption correct that I should first draft a PEP, find a sponsor, and if the PEP gets accepted, implement the feature in typing_extensions and PR in this repo?
Pretty much! But you'd also want to implement it in cpython itself (more important than this repo, because this one is now just for backports) and in at least one major typechecker (e.g., mypy, pyright, pyre).
@jorenham @JelleZijlstra Do you need help with this feature? If no one works on it I can help with it)
@uriyyo I'm currently working on drafting the PEP. From what I've seen, the reference implementations look feasible enough to write myself. But if anything changes, I'll be sure to let you know :)
I'm curious as to why this syntax isn't valid in a function signature that takes a generic i.e.
T = TypeVar("T")
def foo(bar: Sequence[+T]) -> Sequence[+T]: ...
Surely this is still a common use case for variant type vars?
@Gobot1234 Sequence is already covariant. Do you mean something like this instead?
T = TypeVar("T", bound=Sequence)
def foo(bar: T) -> T: ...
@JelleZijlstra Thank you for sponsoring the PEP! I'll be on holiday the coming week, but after that I'll work on expanding the specification so that e.g. class Spam(Sequence[+T], Container[-T]) will not be allowed, like we discussed in the PR. I'm not sure if this issue is the best place for discussing the PEP, so you can also reach me via email (you can find it in the draft PEP).
Sounds good! I'm moving, so I'll also be pretty busy for the next two weeks or so. I'd prefer to discuss the PEP in public. We could also use the issues on your fork of the peps repo (https://github.com/jorenham/peps) to discuss some details.
You'll still need approval from the Steering Council to have me as a sponsor for the PEP. As far as I'm aware this will be the first time the procedure for a non-core dev sponsor will be exercised, so we may have to figure it out as we go along, but I think you should open an issue at https://github.com/python/steering-council/issues asking for permission.
@jorenham I've just started a thread about exactly this on typing-sig. Would would be the best way to provide feedback on the PEP draft?
It's good to see that you see the issue with the current implementation as well @superbobry :). I created an issue on my fork exactly for this a couple of days ago at https://github.com/jorenham/peps/issues/1
Would this syntax be only used in class definitions or it could appear somewhere else?
It comes to my mind that the proposed syntax could be partially incompatible with adding in the future type arithmetic with nice syntax. In particular, if we would introduce the operators +,- at the type level, even if they are restricted to variables representing literal integers and int, it would be unclear what some expressions mean (e.g. A ++ B). And even if we limit variance syntax to class definitions it could be confusing that the + operator means different things.
In particular, if we would introduce the operators +,- at the type level
Later this can also be reused in type-level arithmetics. Like Literal[1] + Literal[2] would resolve into Literal[3]. Or -Literal[2] into Literal[-2]
But, I won't go into this topic yet since it is not directly related.
@fylux The proposed syntax uses unary +/- operators, so it shouldn't interfere much with type level arithmetic, which I assume would mostly use the binary ones. A ++ B looks like a syntax error :)
We instead ended up going with PEP 695, which eliminates the need for explicit variance markers by having type checkers infer variance from use.
This would still be a useful feature, even with PEP695 in place. I outlined 2 main reasons here: https://discuss.python.org/t/proposal-optional-explicit-covariance-contravariance-for-pep-695/35204
