typing
typing copied to clipboard
python
allow `...` in place of generic parameters
from https://github.com/python/mypy/issues/11389
Feature
in kotlin, you can omit a generic from a type annotation when you don't care what its value is:
class Foo<T: Number>
fun foo(value: Foo<*>) {}
more info:
- https://kotlinlang.org/docs/generics.html#star-projections
- https://typealias.com/guides/star-projections-and-how-they-work/
Pitch
-
it's especially useful for types that have multiple bounded generics
i think this could be accomplished by simply allowing
...to be used in place of the genericsThing1 = TypeVar("Thing1", bound=Base1, covariant=True) Thing2 = TypeVar("Thing2", bound=Base2, covariant=True) Thing3 = TypeVar("Thing3", bound=Base3, covariant=True) class ThingWithLotsOfGenerics(Generic[Thing1, Thing2, Thing3]): ... def foo(value: ThingWithLotsOfGenerics[..., ..., ...]) -> None: ... -
Another usage is to ignore variance issues when you don't care about accessing the values.
@dataclass class Box(Generic[T]): t: T def foo(b: Box[...]): print(b) def bar(b: Box[object]): print(b) b = Box(1) foo(b) # no error bar(b) # error, Box[int] incompatible with Box[object]
Alternatives
Use Any
Any removes all type safety so is not a good solution
T = TypeVar("T")
class Foo(Generic[T]):
a: T
def foo(f: Foo[Any]):
f.a = "AMONGUS😳"
f = Foo[int]()
foo(f)
Use object/Never
This doesn't work if your TypeVar is bound, you have to specify the bound, which is non-optimal for many reasons.
class Foo: ...
T = TypeVar("T", bound=Foo, covariant=True)
class Bar(Generic[T]):
...
# error: Type argument "object" of "Bar" must be a subtype of "Foo" [type-var]
def foo(value: Bar[object]) -> None:
...
(also being tracked in KotlinIsland/basedmypy#30 and https://github.com/DetachHead/basedpyright/issues/18)
Another usage is to ignore variance issues when you don't care about accessing the values.
class Box<T>(var t: T)
fun foo(b: Box<*>) = print(b)
fun bar(b: Box<Any>) = print(b)
val b = Box(1)
foo(b) // no error
bar(b) // error, Box<Int> incompatible with Box<Any>
or in python:
@dataclass
class Box(Generic[T]):
t: T
def foo(b: Box[...]): print(b)
def bar(b: Box[object]): print(b)
b = Box(1)
foo(b) # no error
bar(b) # error, Box[int] incompatible with Box[object]
I personally don't see any pros in this version compared to the explicit Any case.
Moreover, we also have implicit Any, so you can write just def foo(value: ThingWithLotsOfGenerics) -> None:
Explicit Any is bad imo(and implicit Any is 100x worse), because it makes the inner of the function unchecked, when it doesn't need to be. Also I always enable all the Any strictness flags, so this would be defeating the purpose and require type: ignore[misc] on a bunch on lines.
T = TypeVar("T", bound=int)
class Foo(Generic[T]):
a: T
def foo(f: Foo[Any]):
f.a = "AMONGUS😳"
The original request was "when you don't care about inner type vars". Now you show cases when you care about them.
In original case it can be implicit or explicit Any.
Thing1 = TypeVar("Thing1", bound=Base1, covariant=True)
Thing2 = TypeVar("Thing2", bound=Base2, covariant=True)
Thing3 = TypeVar("Thing3", bound=Base3, covariant=True)
class ThingWithLotsOfGenerics(Generic[Thing1, Thing2, Thing3]):
...
def foo(value: ThingWithLotsOfGenerics) -> None:
print(value) # you don't care about `value`'s type vars here
When you care about the inner structure or strictness flag is on (I also turn it on for all my projects), you can use:
def foo(f: Foo[T]):
f.a = "AMONGUS😳" # error
(at least with mypy, I am not familiar with other type-checkers' internals)
Sorry for using Kotlin, it's just easier to write for me.
var aList: List<String>? = null
fun foo(someList: List<*>) {
aList = someList // error!
}
If you are taking the type here as Any/dynamic then it's easy to misuse it:
var aList: List<String>? = null
fun foo(someList: List<dynamic>) {
aList = someList // no error! SUS ALERT!
}
When you care about the inner structure or strictness flag is on (I also turn it on for all my projects), you can use:
def foo(f: Foo[T]): f.a = "AMONGUS😳" # error(at least with
mypy, I am not familiar with other type-checkers' internals)
that bounds the function to a generic, which you don't always want, for example:
from typing import TypeVar, Generic
from dataclasses import dataclass
T = TypeVar("T", bound=int, covariant=True)
@dataclass
class Foo(Generic[T]):
value: T
def foo(value: Foo[T] | None = None) -> Foo[T]:
return value or Foo(1) # error: Argument 1 to "Foo" has incompatible type "int"; expected "T"
this doesn't happen with star projections:
class Foo<out T: Number>(value: T)
fun foo(value: Foo<*>?): Foo<*> {
return value ?: Foo(1)
}
fun main() {
val foo: List<Int> = mutableListOf()
}
this is probably a better explanation: https://kotlinlang.org/docs/generics.html#star-projections
This would really save so many boilerplate TypeVars in the project my team is working on. FWIW, Java has this too, ?, called wildcards.
I don't see why you'd need TypeVars if you don't care about the value of the generic parameter. Are you sure your use case corresponds to this feature request?
For what it's worth, I think just using explicit Any works great for this use case (and implicit Any also works). If you hate explicit Any for whatever reason and your type variable is covariant you can use object; if it's contravariant you can use Never.
When you care about the inner structure or strictness flag is on (I also turn it on for all my projects), you can use:
def foo(f: Foo[T]):
f.a = "AMONGUS😳" # error
(at least with mypy, I am not familiar with other type-checkers' internals)
This also works in Pyre and pyright, although pyright additionally issues a warning with this usage that "TypeVar T appears only once in function signature."
So it seems like the only scenario where there isn't already a fine option for this is when all of the following apply: a) you are using pyright and don't want warnings, b) your typevar is invariant (so you can't use object or Never), c) you are using any-strictness and don't want (implicit or explicit) Any either.
If you hate explicit Any for whatever reason
T = TypeVar("T")
class Foo(Generic[T]):
a: T
def foo(f: Foo[Any]):
f.a = "AMONGUS😳"
f = Foo[int]()
foo(f)
If you hate explicit Any for whatever reason and your type variable is covariant you can use object; if it's contravariant you can use Never.
that won't work for bounded TypeVars, which is what my proposed ... syntax aims to solve:
class Foo: ...
T = TypeVar("T", bound=Foo, covariant=True)
class Bar(Generic[T]):
...
# error: Type argument "object" of "Bar" must be a subtype of "Foo" [type-var]
def foo(value: Bar[object]) -> None:
...
Yes, we have a lot of bound invariant TypeVars. Of course, using Any will work for now, but if the code in the future begins to use the type, it will be unchecked.
This also works in Pyre and pyright, although pyright additionally issues a warning with this usage that "TypeVar T appears only once in function signature."
FWIW you can use an object bound to solve this pyright warning
PEP 696 - Type defaults actually solves most of my use cases for this, if I make the default the same as the bound (including using object when it's unbound). It's even less boilerplate too (X vs X[...]). It doesn't work if there's any specified arguments after specified ones though (e.g., X[..., Y]), but I haven't found a use for that in our codebase yet.
Great! I'm going to close this then.
Like Carl and I mention, the situation in which this exact feature request could theoretically have value is just very specific.
Sorry for reviving an old issue, but I'd just like to understand what the recommendation is here, and I'm not sure the type defaults PEP solves my use case.
I'm trying to implement a factory pattern along the lines of:
T = TypeVar("T")
class Runner(Generic[T]):
def __init__(self, arg: str) -> None:
self.arg = arg
def run(self, value: T) -> None:
raise NotImplementedError
def result(self) -> T:
raise NotImplementedError
class IntRunner(Runner[int]):
pass
class StringRunner(Runner[str]):
pass
RunnerFactory = Callable[[str], Runner[T]]
def get_runner(runner_type: str, arg: str) -> Runner[...]:
runners: dict[str, RunnerFactory[...]] = {
"int": IntRunner,
"string": StringRunner,
}
return runners[runner_type](arg)
runner = get_runner("string", "my_arg")
runner.run(10) # this should error! - run should be inferred as (Never) -> None
result = runner.result() # should probably be inferred as object (maybe Never?)
print(runner.arg) # ok! arg doesn't depend on T
Is there a way currently to annotate something like this without losing typing safety? None of the following options work here:
-
RunnerFactory[Any]means we lose type safety sorun(10)type checks even when requesting aRunner[str] -
RunnerFactory[object]fails due to the type invariance (and also it would be wrong anyway -runshouldn't accept anobject) -
RunnerFactory[Never]doesn't work again due to the invariance, though in terms of intent it would have been an acceptable solution if it had worked -
RunnerFactory[T]causes Mypy to complain (probably for good reason - andrunner: Runner[T] = get_runner(...)doesn't make sense anyway sinceTis not bound there)
I am still very much interested in wildcard support!
This was closed? Oh well, I'm still thinking of implementing this in basedmypy.
@arvidfm I don't think this issue would help you.
If you don't want to give up type safety, you can use overloads and literals. You can also use a union type and force caller to assert.
@hauntsaninja I'm pretty sure this issue is what I need. What I described is the exact behaviour of Kotlin's * which this issue was inspired by.
This code works just as expected in Kotlin:
open class Runner<T>(val arg: String) {
fun run(value: T) {
TODO()
}
fun result(): T {
TODO()
}
}
class IntRunner(arg: String) : Runner<Int>(arg)
class StringRunner(arg: String) : Runner<String>(arg)
fun getRunner(runnerType: String, arg: String): Runner<*> {
val runners = mapOf<String, (String) -> Runner<*>>(
"int" to ::IntRunner,
"string" to ::StringRunner,
)
return runners[runnerType]!!(arg)
}
fun main() {
val runner = getRunner("string", "my_arg")
runner.run(52) // ERROR: The integer literal does not conform to the expected type Nothing
val result: Int = runner.result() // ERROR: Type mismatch: inferred type is Any? but Int was expected
println(runner.arg) // ok!
}
Note how T is inferred as Nothing (Kotlin's Never) when used as input, and as Any? (equivalent to object - superclass + nullable) when used as output.
Overloads/unions aren't particularly helpful here since in reality my factory dict contains at least 10+ items and counting, so that would be horrendously verbose. Imagine a union like RunnerFactory[int] | RunnerFactory[str] | RunnerFactory[bool] | ... (RunnerFactory[int | str | bool] wouldn't work here). Even if you created an alias for that it would be a pain to maintain.
Tracking it here: https://github.com/KotlinIsland/basedmypy/issues/30. Not on any radar at the moment though.
Note you can get something vaguely similar by doing:
def get_runner(runner_type: str, arg: str) -> Runner[T]:
runners: dict[str, RunnerFactory[Any]] = {
"int": IntRunner,
"string": StringRunner,
}
return runners[runner_type](arg)
This will make mypy force the caller to add an annotation when doing runner = get_runner(...)
There's nothing stopping the user from just accidentally specifying the wrong annotation though which introduces a lot of room for user error. Plus, really the only safe annotation there would be runner: Runner[Never], which doesn't work if the type variable has constraints, and is still less nice than the behaviour of * in Kotlin (inferring out values as Any?/object).
Could this issue be reopened maybe, since it seems there's still a need that none of the other suggested approaches address?
Sure. Also check out the PEP 696 version: https://pyright-play.net/?code=GYJw9gtgBALgngBwJYDsDmB9ApgDxllAZyTCKiQgTBBigCoAoBgFSgF4pnEsA1AQxAAKAETNhASiYBjADZ9ChKACUArihRYhAcQKakUgNrMAuuIBcDKFagATLMCgYMqJDCeDCWGcAA0UAWhmUIQwIOJQALQAfFAAcqRYFtbJwV7AAHQB7P4gaJbWdg4gah5pfgBufDIqiZzh0XEJQekt%2BVaFUCBYhCoyMKXe9THMza0MsvKKAJIoMKrqmoLzGiAGqDCmo%2BnjcgpQAMqhqGjLi6erISCbUC3bDOcAYnxSMNRw2QDCVXIARjJYBgMl2MfnORmMxiYAFVslwELwBCIocI-IU%2BL0YGxYlhyppJAwOmgsG5igshKSVhh4PCgpc-AFaaEhso1CsDFDjElrBTNIQgjZ9DAgaFQazNE8Xm8DABBFBwCHZADebRSwnWwiCMzmYpAPhVyWEl2OGoOR3Q5z1yQAviqujAVCAUJ0dYQDDyQFTuMZBAF8e7skSSTqREb0CioMIIHAMAEJAx3elSYIAIwABnCAGJYAALJCKQjZsC9GxQTTgEAAQkizqdBaLMhLPyw5BQwDLWBL8iggmxuLCkRi8Q08e6GOyCa6PT6gkzwULxagCHAPz4f3eTZbbZAXU7ijAPwAVlgXt2IHw4Bve5oAPySLq4qqe%2BGCScYyRL9YvnWZXKzsAAayrLIbDAboUAAcloOx4RQEtSE4IA
One alternative to ... could be to use _ instead:
runners: dict[str, RunnerFactory[_]] = {
"int": IntRunner,
"string": StringRunner,
}
which somewhat matches how _ is commonly used in e.g. pattern matching to mean "whatever". There's some related discussion about notation over at the HKT issue - worth noting that the "something that will be supplied elsewhere" part isn't really applicable here. In the context of this issue the wildcard means "I don't care what this is at all".
@arvidfm the issue is that _ is just an ordinary name, and would both need to be initialized beforehand, and somehow understood by the type machinery.
Additionally, _ is often used in the place of unused variables:
a, *_, b = something()
@KotlinIsland That's true, any PEP about this would have to give _ special meaning when used as a type parameter, and I can't say whether there's any precedence for that. ~~I also wonder if there are weird values that a user might assign to _ that would break evaluation~~*. Though with deferred evaluation of annotations I don't think it would be unreasonable to have the annotation parser interpret _ differently from other scopes. Might be as easy as just injecting a variable _ with some special WildcardType value into the local scope when evaluating the annotation.
*Edit: Of course there are, assigning any value would break evaluation when not using deferred evaluation