canonical
Should we vendor typing_extensions.NotRequired?
The code is really simple:
https://github.com/python/typing_extensions/blob/70cec91bec65155dc339d631ede2a933582558df/src/typing_extensions.py#L2220-L2256
We're using:
-
Requiredin ops/charm.py -
NotRequiredin ops/pebble.py -
NotRequiredin test/fake_pebble.py
What's the motivation for this? Are they being removed from typing_extensions, or is there some other reason?
The motivation is to avoide the if TYPE_CHECKING: block.
The Required/NotRequired are used in TypedDict definitions, which means that TypedDict's can't be used without quotes. Granted, those typed dicts are internal.
I think it may partially help with the following comment for public typed dicts:
# In Python 3.11+ 'services' and 'labels' should be NotRequired, and total=True.
The latter could be an improvement for our library users, maybe?
Possibly related: #1336
We're uncertain whether this will actually work once they're vendored with Pyright. Investigate on a rainy day next cycle.
I suspect that this can't be vendored, given the following quote from typing-extensions:
"typing_extensions is treated specially by static type checkers such as mypy and pyright. Objects defined in typing_extensions are treated the same way as equivalent forms in typing."
I haven't used typing_extensions much, but do we have to use its constructs with if TYPE_CHECKING? I see that we have used it this way, for example in ops/pebble.py:
if TYPE_CHECKING:
from typing_extensions import NotRequired
As far as I can tell, typing_extensions currently supports Python versions 3.8 and higher, so do we really need if TYPE_CHECKING?
Maybe the outcome (with a little bit of further rainy day investigation) will be closing this and making a new issue: let's use Required/NotRequired from typing_extensions to improve our TypeDicts without if TYPE_CHECKING.
I haven't used
typing_extensionsmuch, but do we have to use its constructs withif TYPE_CHECKING?
The advantage of this is that typing_extensions is not a runtime dependency, only a test time one.
That makes sense. With from __future__ import annotations we can put the typing_extensions imports behind if TYPE_CHECKING but use its constructs in annotations freely outside if TYPE_CHECKING. Without it, we can still put just the imports inside if TYPE_CHECKING and have the definitions just use string annotations.
Since we're going to go with from __future__ import annotations in future, I think we can close this in favour of (#1336), but I'll hold off on doing so in case anyone wants to investigate vendoring further.
typing_extensions is treated specially by static type checkers such as mypy and pyright. Objects defined in typing_extensions are treated the same way as equivalent forms in typing.
perhaps that means we can't naively vendor some bits from typing_extensions. We could still play with a vendored module that happens to be called "typing_extensions" but I doubt that's worth the hassle, as we'd have to make a funky import to ensure we don't break charms that imports real type_extensions and thus has real module in their virtual env(s).
my take: either real typing_extensions or nothing.
The types using NotRequired are _Private in ops/pebble.py. Even if we wanted to make some of them public, we could probably use strings around our type annotations. But we don't think anything is needed here now.
We discussed this in the Charm-Tech daily and decided that we're not going to do this, per the discussion items above.