typeshed icon indicating copy to clipboard operation
typeshed copied to clipboard

Consider adding `__slots__` to stubs?

Open sobolevn opened this issue 3 years ago • 10 comments

Since mypy now supports __slots__ checking, we can add __slots__ to stubs. This will allow us to correctly raise Something is not in __slots__ when assigning extra attributes.

Right now there are no __slots__ defined: https://github.com/python/typeshed/search?q=slots

If this is something we are going for, there are several things to consider:

  1. Adding this feature to stubtest. It can check that __slots__ contents do match. Probably behind a feature flag. We can enable it for stdlib only

  2. Should we allow skipping _protected attributes? For example, if some class is defined as

    class A:
       __slots__ = ('_prop',)
    

    Should we allow this stub?

    class A:
        __slots__ = ()
    

sobolevn avatar Oct 03 '22 17:10 sobolevn

  1. Adding this feature to stubtest. It can check that __slots__ contents do match. Probably behind a feature flag. We can enable it for stdlib only

It's currently included in stubtest's list of always-ignored dunders inside classes: https://github.com/python/mypy/blob/dc5c299aa190949f2300b163ccc10257a779006d/mypy/stubtest.py#L1280-L1281.

AlexWaygood avatar Oct 03 '22 17:10 AlexWaygood

Technically it's not always ignored, it's "ignorable". Aka if the runtime has __slots__, stubtest won't currently enforce that the stub has __slots__. But if the stub does have __slots__, stubtest will make sure they match the runtime.

hauntsaninja avatar Oct 03 '22 17:10 hauntsaninja

I agree that adding __slots__ is useful in stubs.

For your question 2, I think it's best to add protected attributes. That way, we can just have the slots definition match exactly between runtime and the stub, which is easier to test and review.

JelleZijlstra avatar Oct 03 '22 17:10 JelleZijlstra

Ok, I will start with stubtest PR! 👍

sobolevn avatar Oct 03 '22 18:10 sobolevn

to correctly raise Something is not in slots when assigning extra attributes.

Can you clarify this? Type checkers complain about assigning non-existent attributes anyway, unless it's a subclass and then the __slots__ of the parent class doesn't matter anyway at runtime (unless the subclass defines its own __slots__, in that case the fields that are actually accessible are Parent.__slots__ + Child.__slots__).

Akuli avatar Oct 03 '22 22:10 Akuli

As @Akuli says, I don't think adding __slots__ to the stub helps much with whether or not type checkers will validate attribute assignments.

Where it might help is in code like this:

>>> class A:
...     __slots__ = ["foo"]
... 
...     
>>> class B:
...     __slots__ = ("bar",)
... 
...     
>>> A.__slots__ + B.__slots__
Traceback (most recent call last):
  File "<pyshell#6>", line 1, in <module>
    A.__slots__ + B.__slots__
TypeError: can only concatenate list (not "tuple") to list
>>> class C(A, B): ...
... 
Traceback (most recent call last):
  File "<pyshell#9>", line 1, in <module>
    class C(A, B): ...
TypeError: multiple bases have instance lay-out conflict

Adding __slots__ to the stub could plausibly help type checkers catch both of those exceptions.

AlexWaygood avatar Oct 03 '22 22:10 AlexWaygood

Sorry, an example was needed 🙂

Mypy has a special feature to check attribute definitions to be compatible with __slots__: https://github.com/python/mypy/blob/dc5c299aa190949f2300b163ccc10257a779006d/mypy/checker.py#L3196-L3225 Tests can be found here: https://github.com/python/mypy/blob/master/test-data/unit/check-slots.test

So, let's say we have a definition of SomeStub type in typeshed, which does not have __slots__ defined:

class SomeStub: ...  # defined in typeshed


class A(SomeStub):  # user's code
    __slots__ = ("b",)

    def __init__(self) -> None:
        # All three will pass, because `SomeStub` parent has `__dict__`:
        self.a = 1
        self.b = 2
        self._one = 1

Compare it to the next case, where we have __slots__ defined:

class SomeStubWithSlots:  # defined in typeshed
    __slots__ = ("a",)


class A(SomeStubWithSlots):  # user's code
    __slots__ = ("b",)

    def __init__(self) -> None:
        self.a = 1
        self.b = 2
        self._one = 1  # E: Trying to assign name "_one" that is not in "__slots__" of type "ex.A"

sobolevn avatar Oct 04 '22 08:10 sobolevn

I now see how it's useful. It is an improvement, but also not super important because new attributes are usually introduced in __init__, so the user's code would fail every time A is instantiated, not only in some obscure corner case. It's also somewhat rare to use __slots__ in user code.

Akuli avatar Oct 04 '22 09:10 Akuli

It's also somewhat rare to use slots in user code.

This is shocking to me :) So many people ignore this awesome feature for some reason.

sobolevn avatar Oct 04 '22 12:10 sobolevn

#14611 adds __slots__ to almost all of the standard library. I might do the third-party stubs soon too.

JelleZijlstra avatar Aug 21 '25 14:08 JelleZijlstra