typing
typing copied to clipboard
python
Introduce an Intersection
This question has already been discussed in #18 long time ago, but now I stumble on this in a practical question: How to annotate something that subclasses two ABC's. Currently, a workaround is to introduce a "mix" class:
from typing import Iterable, Container
class IterableContainer(Iterable[int], Container[int]):
...
def f(x: IterableContainer) -> None: ...
class Test(IterableContainer):
def __iter__(self): ...
def __contains__(self, item: int) -> bool: ...
f(Test())
but mypy complains about this
error: Argument 1 of "__contains__" incompatible with supertype "Container"
But then I have found this code snippet in #18
def assertIn(item: T, thing: Intersection[Iterable[T], Container[T]]) -> None:
if item not in thing:
# Debug output
for it in thing:
print(it)
Which is exactly what I want, and it is also much cleaner than introducing an auxiliary "mix" class. Maybe then introducing Intersection is a good idea, @JukkaL is it easy to implement it in mypy?
Mypy complains about your code because __contains__ should accept an argument of type object. It's debatable whether this is the right thing to do, but that's how it's specified in typeshed, and it allows Container to be covariant.
I'm worried that intersection types would be tricky to implement in mypy, though conceptually it should be feasible. I'd prefer supporting structural subtyping / protocols -- they would support your use case, as IterableContainer could be defined as a protocol (the final syntax might be different):
from typing import Iterable, Container
class IterableContainer(Iterable[int], Container[int], Protocol):
...
def f(x: IterableContainer) -> None: ...
class Test:
def __iter__(self): ...
def __contains__(self, item: int) -> bool: ...
f(Test()) # should be fine (except for the __contains__ argument type bit)
It would be really cool to implement protocols. Still, in this case intersection could be added as a "syntactic sugar", since there would be a certain asymmetry: Assume you want a type alias for something that implements either protocol, then you write:
IterableOrContainer = Union[Iterable[int], Container[int]]
But if you want a type alias for something that implements both, you would write:
class IterableContainer(Iterable[int], Container[int], Protocol): ...
I imagine such asymmetry could confuse a novice. Intersection could then be added (very roughly) as:
class _Intersection:
def __getitem__(self, bases):
full_bases = bases+(Protocol,)
class Inter(*full_bases): ...
return Inter
Intersection = _Intersection()
then one could write:
IterableContainer = Intersection[Iterable[int], Container[int]]
Intersection[...] gets tricky once you consider type variables, callable types and all the other more special types as items. An intersection type that only supports protocols would be too special purpose to include, as it's not even clear how useful protocols would be.
I understand what you mean. That could be indeed tricky in general case.
Concerning protocols, I think structural subtyping would be quite natural for Python users, but only practice could show whether it will be useful. I think it will be useful.
This keeps coming up, in particular when people have code that they want to support both sets and sequences -- there is no good common type, and many people believe Iterable is the solution, but it isn't (it doesn't support __len__).
I think Intersection is a very natural thing (at least if one thinks about types as sets, as I usually do). Also, it naturally appears when one wants to support several ABCs/interfaces/protocols.
I don't think that one needs to choose between protocols and Intersection, on the contrary they will work very well in combination. For example, if one wants to have something that supports either "old-style" reversible protocol (i.e. has __len__ and __iter__ methods) or "new-style" (3.6+) reversible protocol (i.e. has __reversed__ method), then the corresponding type is Union[Reversible, Intersection[Sized, Iterable]].
It is easy to add Intersection to PEP 484 (it is already mentioned in PEP 483) and to typing.py, the more difficult part is to implement it in mypy (although @JukkaL mentioned this is feasible).
For cross-reference from #2702, this would be useful for type variables, e.g. T = TypeVar('T', bound=Intersection[t1, t2]).
Intersection[FooClass, BarMixin] is something I found myself missing today
If we had an intersection class in typing.py, what would we call it?
Intersection is linguistically symmetric with Union, but it's also rather long.
Intersect is shorter, but it's a verb. Meet is the type-theoretic version and also nice and short, but, again, you'd expect Union to be called Join if you call Intersection Meet.
Just as a random idea I was thinking about All (it would be more clear if Union would be called Any, but that name is already taken). In general, I don't think long name is a big issue, I have seen people writing from typing import Optional as Opt or even Optional as O depending on their taste. Also generic aliases help in such cases:
T = TypeVar('T')
CBack = Optional[Callable[[T], None]]
def process(data: bytes, on_error: CBack[bytes]) -> None:
...
I just opened #483 hoping for exactly the same thing. I literally named it the same. I would be all for Intersection or All to allow to require a list of base classes.
Requests for Intersection appear here and there, maybe we should go ahead and support it in mypy? It can be first put in mypy_extensions or typing_extensions. It is a large piece of work, but should not be too hard. @JukkaL @gvanrossum what do you think?
I think we should note the use cases but not act on it immediately -- there are other tasks that IMO are more important.
@gvanrossum
I think we should note the use cases but not act on it immediately -- there are other tasks that IMO are more important.
OK, I will focus now on PEP 560 plus related changes to typing. Then later we can get back to Intersection, this can be a separate (mini-) PEP if necessary.
Btw, looking at the milestone "PEP 484 finalization", there are two important issues that need to be fixed soon: https://github.com/python/typing/issues/208 (str/bytes/unicode) and https://github.com/python/typing/issues/253 (semantics of @overload). The second will probably require some help from @JukkaL.
I agree that now's not the right time to add intersection types, but it may make sense later.
(been redirected here from the mailing list)
I think the Not type needs to be added in addition to Intersection:
Intersection[Any, Not[None]]
Would mean anything but None.
How about the expression Type1 | Type2 and Type1 & Type2 alternative to Union and Intersection respectively.
example:
x: int & Const = 42
@rnarkk these have already been proposed many times, but have not been accepted.
The Not special form hasn't been proposed before to my knowledge. I suppose you could equivalently propose a Difference[Any, None] type.
What's the use case for that? It's not something I've ever missed in a medium-sized typed codebase at work and in lots of work on typeshed.
@JelleZijlstra My specific case was to specify Any but None.
Difference and other set-alike operators can be expressed using Union, Intersection and Not.
I don't think Not[T] fits in with the rest of the type system; it sounds more like something you'd want to do at runtime, and then specifically only for "not None".
This keeps coming up, in particular when people have code that they want to support both sets and sequences -- there is no good common type, and many people believe Iterable is the solution, but it isn't (it doesn't support
__len__).
@gvanrossum So what is the solution? (see also my stackoverflow question)
Have you tried defining a custom protocol which subclasses the relevant protocols? Or you can explicitly define all the methods you care about in a custom protocol.
It seems that protocols are not available in typing under 3.6. The following code works and has no warnings in mypy.
from typing_extensions import Protocol
class SizedIterable(Protocol):
def __len__(self):
pass
def __iter__(self):
pass
def foo(some_thing: SizedIterable):
print(len(some_thing))
for part in some_thing:
print(part)
foo(['a', 'b', 'c'])
Thanks!
Cross-posting from python/mypy#3135, as a use case for Intersection that I don't believe is possible right now: class-decorators that add methods.
class FooBar(Protocol):
def bar(self) -> int:
return 1
T = TypeVar("T", bound=Type)
def add_bar(cls: T) -> Intersection[FooBar, T]:
def bar(self) -> int:
return 2
cls.bar = bar
@add_bar
class Foo: pass
Foo().bar()
Now I also understand that mypy doesn't support class decorators yet, but having a way to describe them correctly, and having mypy support them seem 2 different issues.
This is also highly needed for mixin classes to annotate self-type, for cases where mixin can only be mixed to some type or even also require another mixins.
class Base:
def do(self, params: dict):
raise NotImplementedError
class Mixin:
def _auxilary(self, params: dict) -> dict:
return dict(params, foo='bar')
def do(self: Intersection['Mixin', Base], params: dict):
super().do(self._auxilary(params)) # so IDE shows no warning about `.do` or `._auxilary`
Any workaround for this @gvanrossum
Cross-posting from python/mypy#3135, as a use case for Intersection that I don't believe is possible right now: class-decorators that add methods.
class FooBar(Protocol): def bar(self) -> int: return 1 T = TypeVar("T", bound=Type) def add_bar(cls: T) -> Intersection[FooBar, T]: def bar(self) -> int: return 2 cls.bar = bar @add_bar class Foo: pass Foo().bar()Now I also understand that mypy doesn't support class decorators yet, but having a way to describe them correctly, and having mypy support them seem 2 different issues.
Any update on this? Is it still on the road map? Mixins are a core feature of the language, I don't think this is an edge case. I know it can be worked around by defining base classes, but it means adding unnecessary boilerplate and overhead just to please type checkers.
Thanks for all the great work!
