Implement a way to specify typevar of Selector
Background
My usual usecase of Selector is when I have an interface with multiple implementations:
class Animal:
def __init__(self) -> None:
pass
class Dog(Animal):
pass
class Cat(Animal):
pass
And then the container:
class Container(containers.DeclarativeContainer):
get_dog = providers.Singleton(Dog)
get_cat = providers.Singleton(Cat)
get_animal = providers.Selector(lambda: random.choice(["cat", "dog"]), cat=get_cat, dog=get_dog)
Usage:
def main() -> None:
container = Container()
animal = container.get_animal()
reveal_type(animal) # Revealed type is Any
Desired behaviors
Ideally, Selector should:
- Allow users to specify type:
get_animal = providers.Selector[Animal](lambda: random.choice(["cat", "dog"]), cat=get_cat, dog=get_dog) - Automatically infer from the kwargs of providers: cat and dog are both animals, so get_animal is
Selector[Animal]
Possible solutions I have explored
TL;DR:
- Solution 1: gives desired behavior 1, but forces users to manually-type
- Solution 2: gives desired behavior 1, but only works on Python 3.13 or above
- Solution 3: gives both desired behavior 1 and 2, but requires changing
Providerto be covariant
-
Simply making
Selectorgeneric toT = TypeVar("T")Stub
class Selector(Provider[T]): def __init__( self, selector: Optional[_Callable[..., Any]] = None, **providers: Provider ): ...Result
import random from typing import reveal_type from dependency_injector import containers, providers class Animal: def __init__(self) -> None: pass class Dog(Animal): pass class Cat(Animal): pass class Container(containers.DeclarativeContainer): get_dog = providers.Singleton(Dog) get_cat = providers.Singleton(Cat) # Not providing type results in error: 'Need type annotation for "get_animal" [var-annotated]' get_animal = providers.Selector(lambda: random.choice(["cat", "dog"]), cat=get_cat, dog=get_dog) get_animal_with_type = providers.Selector[Animal](lambda: random.choice(["cat", "dog"]), cat=get_cat, dog=get_dog) def main() -> None: container = Container() animal = container.get_animal() animal_with_type = container.get_animal_with_type() reveal_type(animal) # Revealed type is Any reveal_type(animal_with_type) # Revealed type is Animal if __name__ == "__main__": main()This gives us desired behavior 1, but it forces users to always manually specify the type, otherwise var-annotated error is raised.
Also, this doesn't result in desired behavior 2.
-
Another option is to just add default to the typevar
Stub
T_Any = TypeVar("T_Any", default=Any) class Selector(Provider[T_Any]): def __init__( self, selector: Optional[_Callable[..., Any]] = None, **providers: Provider ): ...The result is the same as solution 1, but without the var-annotated error. However, typevar default is only available in Python 3.13 onwards https://peps.python.org/pep-0696/, so we'll have to check Python version on the stub, something along the lines of
if sys.version_info > (3, 13): T = TypeVar("T", default=Any) else: T = TypeVar("T") -
Finally, we can make
Providergeneric to a covariant typevar.Making provider generic to a covariant typevar allow mypy type auto-inference to work.
Stubs
T = TypeVar("T") TT = TypeVar("TT") T_Co = TypeVar("T_Co", covariant=True) P = TypeVar("P", bound="Provider") BS = TypeVar("BS", bound="BaseSingleton") class Provider(Generic[T_Co]): def __init__(self) -> None: ... @overload def __call__(self, *args: Injection, **kwargs: Injection) -> T_Co: ... @overload def __call__(self, *args: Injection, **kwargs: Injection) -> Awaitable[T_Co]: ... def async_(self, *args: Injection, **kwargs: Injection) -> Awaitable[T_Co]: ... . . . def provider(self) -> Provider[T_Co]: ... @property def provided(self) -> ProvidedInstance[T_Co]: ... . . . . . . class Selector(Provider[T_Co]): def __init__( self, selector: Optional[_Callable[..., Any]] = None, **providers: Provider[T_Co] ): ...Result
import random from typing import reveal_type from dependency_injector import containers, providers class Animal: def __init__(self) -> None: pass class Dog(Animal): pass class Cat(Animal): pass class Container(containers.DeclarativeContainer): get_dog = providers.Singleton(Dog) get_cat = providers.Singleton(Cat) get_animal = providers.Selector(lambda: random.choice(["cat", "dog"]), cat=get_cat, dog=get_dog) def main() -> None: container = Container() animal = container.get_animal() reveal_type(animal) # Revealed type is Animal if __name__ == "__main__": main()Providerneeds to be covariant in order for this to work, so thatProvider[Dog]andProvider[Cat]is a subclass ofProvider[Animal].
Just looking at the stubs, I see no reason why Provider shouldn't be covariant, as it doesn't take typevar as parameter of its methods.
Though I'd like to see what the maintainers think before working on a PR.
Though I'd like to see what the maintainers think before working on a PR.
I think better type support is always welcome.
- Simply making Selector generic to T = TypeVar("T")
Our current class Selector(Provider[Any]) is a lazy way to avoid typing issues, which is sub-optimal. Making it generic is a an option, but as you've already mentioned, this will force users to manually specify types, so this option is no-go.
- Another option is to just add default to the typevar.
This one looks safest in terms of backward compatibility. There are also places in our (and others) code where Provider is used without explicit type specification, so this change is good on its own either way.
Note: We already depend on typing-extensions for Python<3.11, I think it would be better to extend it to Python<3.13 and import TypeVar from typing_extensions instead of doing version checks.
- Finally, we can make Provider generic to a covariant typevar.
This sounds like a good idea, but we have to get rid of Provider without explicit type specification first to be sure it won't break typing elsewhere.
Let's do option 2 first and after that another PR for option 3. I want to run mypy against my codebases for changes from option 3 in isolation to see if there are any breaking changes.
Let's do option 2 first and after that another PR for option 3. I want to run mypy against my codebases for changes from option 3 in isolation to see if there are any breaking changes.
Sounds good! Will go ahead with the PR for option 2.
Also, agreed on the point with typing-extensions instead of version checks.
I realized we have a similar problem with Aggregate too. In this case, the stub declares the type as if Provider is already covariant, but it's wrong since it's still invariant.
class Aggregate(Provider[T]):
def __init__(
self,
provider_dict: Optional[_Dict[Any, Provider[T]]] = None,
# Mypy cannot infer "most specific common base type" for T here since Provider is invariant
**provider_kwargs: Provider[T],
): ...
Trying with the example from the documentation page here
# main.py
from dependency_injector import containers, providers
class ConfigReader:
def __init__(self, path):
self._path = path
def read(self):
print(f"Parsing {self._path} with {self.__class__.__name__}")
...
class YamlReader(ConfigReader):
...
class JsonReader(ConfigReader):
...
class Container(containers.DeclarativeContainer):
config_readers = providers.Aggregate(
yaml=providers.Factory(YamlReader),
json=providers.Factory(JsonReader),
)
Mypy result:
(.venv) :~/src/personal/python-dependency-injector$ mypy main.py
main.py:25: error: Cannot infer value of type parameter "T" of "Aggregate" [misc]