mypy
mypy copied to clipboard
python
abstract dataclass inheritance gives `Only concrete class can be given where "Type[Abstract]" is expected`
mypy 0.620, --strict.
Sample code:
import abc
from dataclasses import dataclass
@dataclass # error: Only concrete class can be given where "Type[Abstract]" is expected
class Abstract(metaclass=abc.ABCMeta):
a: str
@abc.abstractmethod
def c(self) -> None:
pass
@dataclass
class Concrete(Abstract):
b: str
def c(self) -> None:
pass
instance = Concrete(a='hello', b='world')
Note that this error only occurs if at least one abstract method is defined on the abstract class. Removing the abstract method c (or making it non-abstract) makes the error go away.
What if you remove the meta class?
On Wed, Jul 18, 2018 at 9:01 PM Jonas Obrist [email protected] wrote:
mypy 0.620, --strict.
Sample code:
import abcfrom dataclasses import dataclass
@dataclass # error: Only concrete class can be given where "Type[Abstract]" is expectedclass Abstract(metaclass=abc.ABCMeta): a: str
@abc.abstractmethod def c(self) -> None: pass@dataclassclass Concrete(Abstract): b: str
def c(self) -> None: passinstance = Concrete(a='hello', b='world')
Note that this error only occurs if at least one abstract method is defined on the abstract class. Removing the abstract method c (or making it non-abstract) makes the error go away.
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/python/mypy/issues/5374, or mute the thread https://github.com/notifications/unsubscribe-auth/ACwrMmnqdQS-PT-Ppma35aXOsPpD6X7vks5uIASAgaJpZM4VVt0v .
-- --Guido (mobile)
Removing the metaclass still triggers it. Minimal code to get that error:
import abc
from dataclasses import dataclass
@dataclass
class Abstract:
@abc.abstractmethod
def c(self) -> None:
pass
The underlying cause is this:
from typing import Type, TypeVar
from abc import abstractmethod
T = TypeVar('T')
class C:
@abstractmethod
def f(self) -> None: ...
def fun(x: Type[T]) -> Type[T]: ...
fun(C)
A possible simple solution is to type dataclass in typeshed as roughly (S) -> S with S = TypeVar('S', bound=Type[Any]). Another option is to consider relaxing the restriction that mypy has for Type[T] with abstract classes, but I am not sure where to draw the line.
Lifting this constraint would be great for fetching implementations to interfaces (abstracts/protocols).
T_co = TypeVar('T_co', covariant=True)
def register(interface: Type[T_co], implementation: Type[T_co]) -> Type[T_co]: ...
def inject(interface: Type[T_co]) -> T_co: ...
Just a bit of context why this is prohibited: mypy always allows instantiation of something typed as Type[...], in particular of function arguments, for example:
from typing import Type, TypeVar
from abc import abstractmethod
T = TypeVar('T')
class C:
@abstractmethod
def f(self) -> None: ...
def fun(x: Type[T]) -> T: ...
return x() # mypy allows this, since it is a very common pattern
fun(C) # this fails at runtime, and it would be good to catch it statically,
# what we do currently
I am still not sure what is the best way to proceed here. Potentially, we can only prohibit this if original function argument (or its upper bound) is abstract (so function actually expects implementations to be passed). This will be a bit unsafe, but more practical, as all examples provided so far would pass.
Another option is to consider relaxing the restriction that mypy has for
Type[T]with abstract classes, but I am not sure where to draw the line.
I'm leaning towards allowing abstract classes to be used without restriction. Type[x] is already unsafe since __init__ can be overriden with a different signature, and I expect that the extra unsafety from allowing abstract classes would be marginal in practice. We just need to document the potential issues clearly.
I'm leaning towards allowing abstract classes to be used without restriction. [...] We just need to document the potential issues clearly.
This will be false negative, so probably OK.
There is no concrete work planned for this issue yet, we are focusing on high-priority issues.
I'm having a similar error and I don't know if it's the same problem or a different, related problem (or perhaps I'm doing something wrong), related to abstract classes and Attrs.
I have this abstract class: https://github.com/eventbrite/pymetrics/blob/ab608978/pymetrics/recorders/base.py#L38-L39
Then I have this constant:
_DEFAULT_METRICS_RECORDER = NoOpMetricsRecorder() # type: MetricsRecorder
And this Attrs class:
@attr.s
@six.add_metaclass(abc.ABCMeta)
class RedisTransportCore(object):
...
metrics = attr.ib(
default=_DEFAULT_METRICS_RECORDER,
validator=attr.validators.instance_of(MetricsRecorder),
) # type: MetricsRecorder
On the line with attr.validators.instance_of, I'm getting this error:
error: Only concrete class can be given where "Type[MetricsRecorder]" is expected
What's interesting is that this only happens with today's released Attrs 19.2.0 and last week's released MyPy 0.730. If I downgrade to Attrs 19.1.0 and keep the latest MyPy, the error goes away. If I downgrade to MyPy 0.720 and keep the latest Attrs, the error goes away.
I'm able to get around the problem by adding # type: ignore at the end of the line with attr.validators.instance_of, so I'm good for now, but I would like to know if I need to file a separate issue or if this is related or if this is a bug with Attrs.
This is really sad, given that dataclasses are being pushed as a go-to class container in 3.7+, having abstract dataclasses is not such a rare occasion. But mypy in the current state basically prohibits using it.
Given that this issue is 1.5 years old, wonder if there's any known workarounds?
Marking as high priority, since there seems to be quite a lot of interest in getting this fixed.
Depending on your needs, for a temporary workaround, you can do the following, which type checks with --strict.
import abc
import dataclasses
class AbstractClass(abc.ABC):
"""An abstract class."""
@abc.abstractmethod
def method(self) -> str:
"""Do something."""
@dataclasses.dataclass
class DataClassMixin:
"""A dataclass mixin."""
spam: str
class AbstractDataClass(DataClassMixin, AbstractClass):
"""An abstract data class."""
@dataclasses.dataclass
class ConcreteDataClass(AbstractDataClass):
"""A concrete data class."""
eggs: str
def method(self) -> str:
"""Do something."""
return "{} tastes worse than {}".format(self.spam, self.eggs)
print(ConcreteDataClass("spam", "eggs").method())
# -> "spam tastes worse than eggs"
Similar to @cybojenix, we would be very happy if mypy could support service locator patterns similar to the following (the code for which works, but does not pass mypy type checks for want of a concrete class.)
from typing import Any, Dict, TypeVar, Type
from typing_extensions import Protocol
ServiceType = TypeVar("ServiceType")
registrations: Dict[Any, Any] = {}
def register(t: Type[ServiceType], impl: ServiceType) -> None:
registrations[t] = impl
def locate(t: Type[ServiceType]) -> ServiceType:
return registrations[t]
class ThingService(Protocol):
def serve(str) -> str:
pass
class ThingImpl(ThingService):
def serve(str) -> str:
return "fish"
register(ThingService, ThingImpl())
thing: ThingService = locate(ThingService)
print(thing.serve())
Interestingly, the given rationale of not allowing abstract classes ("mypy always allows instantiation of something typed as Type[...], in particular of function arguments"), does not seem to match its implementation, as classes based on abc.ABC cannot be instantiated, but code using only this actually pass the mypy check, whereas classes with @abstractmethods can be instantiated, but classes using those trigger the false positives discussed in this issue.
To elaborate (and give another example to possibly test a fix against later), the following code uses only @abstractmethods and triggers the false positive:
from abc import ABC, abstractmethod
from typing import List, Type, TypeVar
AB = TypeVar("AB", "A", "B") # Could be A or B (Union)
class A:
@abstractmethod
def snork(self):
pass
class B:
@abstractmethod
def snork(self):
pass
class oneA(A):
pass
class anotherA(A):
pass
class someB(B):
pass
def make_objects(of_type: Type[AB]) -> List[AB]:
return [cls() for cls in of_type.__subclasses__()]
if __name__ == "__main__":
print(make_objects(A))
print(make_objects(B))
Removing the two @abstractmethod decorators and changing class A: and class B: to class A(ABC): and class B(ABC): actually passes mypy's check without errors.
Similar to @ToBeReplaced, workaround I've been using:
@dataclass
class _AbstractDataclass:
id: int
class AbstractDataclass(_AbstractDataclass, abc.ABC):
@abc.abstractmethod
def c(self) -> None:
pass
Another use-case which also doesn't work due to this limitation is the adapter pattern using typing Protocols.
i.e.:
from typing import Protocol, TypeVar, Type
T = TypeVar('T')
class IFoo(Protocol):
def foo(self) -> str:
pass
class Adaptable(object):
def get_adapter(self, class_: Type[T]) -> T:
...
def some_call(adaptable: Adaptable) -> None:
foo = adaptable.get_adapter(IFoo)
This is over 2 years old, and 0.800 just started doing a better job at finding these cases which resulted with a wall of errors in my project. Thanks, mypy. :(
@mkielar could you give concrete examples of code that started erroring? Perhaps we can easily fix the regression.
@JelleZijlstra
My production code cut to be smaller:
import dataclasses as dc
from abc import ABCMeta, abstractmethod
@dc.dataclass(frozen=True)
class CanaryConfiguration(metaclass=ABCMeta):
@abstractmethod
def is_fast_track(self) -> bool:
"""
"""
@dc.dataclass(frozen=True)
class CanaryConfigurationEcs(CanaryConfiguration):
def is_fast_track(self) -> bool:
return True
@dc.dataclass(frozen=True)
class Canary:
configuration: CanaryConfiguration
def test_me() -> bool:
c: Canary = Canary(CanaryConfigurationEcs())
return c.configuration.is_fast_track()
Console output:
→ mypy --version
mypy 0.800
→ mypy test.py
test.py:5: error: Only concrete class can be given where "Type[CanaryConfiguration]" is expected
Found 1 error in 1 file (checked 1 source file)
In actual code it additionally complains that the return statement in test_me() returns Any instead of bool but somehow if I cut the business logic I can no longer reproduce it.
Adding one more (hopefully) valid use case
from dataclasses import dataclass, fields, is_dataclass
from typing import Protocol, Optional, cast
@dataclass
class MyConfig:
v1: str
v2: 'Optional[str]' = None
@dataclass
class GlobalConfig:
MyConfig: MyConfig
class SourceList(Protocol):
def get(self, key: str) -> str:
pass
class ConfigProtocol(Protocol): # this is one option, but just being able to type check dataclass would be fine
__dataclass_fields__: dict
class Configuration:
def __init__(
self,
source_list: SourceList,
config_class: ConfigProtocol = GlobalConfig
):
self._config_sources = source_list
for f in fields(self._config_class):
if is_dataclass(f.type):
setattr(
self,
f.name,
Configuration(
source_list=source_list,
config_class=cast(ConfigProtocol, f.type)
)
)
def __getattr__(self, name):
if value := self._config_sources.get(name):
return value
raise AttributeError()
mypy output:
> mypy --version
mypy 0.800
> mypy test.py
test.py:24: error: Incompatible default for argument "config_class" (default has type "Type[GlobalConfig]", argument has type "ConfigProtocol")
Found 1 error in 1 file (checked 1 source file)
Define dataclasses for config objects in modules and sub-modules, which can be instantiated and used directly, or as shown here, used as templates to build a more complex config class.
Minimal example similar to @mkielar's but with structural subtyping
from dataclasses import dataclass
from typing import Protocol
@dataclass(frozen=True)
class Large:
x: int
y: int
@dataclass(frozen=True)
class Small(Protocol):
x: int
def f(x: Small) -> Small:
...
f(Large(1, 2))
It starts to work when you remove (frozen=True). What's strange, is that it does not work with @dataclass() but works with @dataclass, while they are equivalent according to docs
@Tomatosoup97, the fact that mypy inconsistently flags this as an error based on the form of dataclass looks like a bug that should probably be reported separately.
I wouldn't expect your sample to type check with any form of dataclass because Small and Large have incompatible __init__ types. With dataclass, the __init__ method is synthesized based on the defined dataclass fields, and the __init__ method needs to be considered when checking for structural subtype compatibility. Because of this, Large is not type compatible with the Small protocol.
@erictraut right, thanks for noting this, I have reported it separately as I didn't find any already existing issue: #10849
As for the expected failure - do you know any other way then to do achieve such structural subtyping, focused on fields? I guess I could just define a bunch of @propertys but that seems a bit cumbersome
@Tomatosoup97, you could remove the @dataclass designator from the protocol. That type checks without errors if the dataclass isn't frozen.
@dataclass
class Large:
x: int
y: int
class Small(Protocol):
x: int
If the dataclass is frozen, mypy will fail the protocol check because x is read-only in the dataclass but not in the protocol. I would expect that you could specify the field to be Final in the protocol, indicating that it's read-only. That should pass type checking, but mypy emits an error in this case still. Pyright is fine with this.
@dataclass(frozen=True)
class Large:
x: int
y: int
class Small(Protocol):
x: Final[int] = 0
This documentation is incomplete -- it doesn't mention that the Type annotation must represent a concrete class (and not an abstract one): https://mypy.readthedocs.io/en/stable/kinds_of_types.html#the-type-of-class-objects
There are multi-year-old StackOverflow questions on this subject: https://stackoverflow.com/questions/48349054/how-do-you-annotate-the-type-of-an-abstract-class-with-mypy
I personally have a case like the one mentioned on StackOverflow:
from typing import Type, TypeVar
_T = TypeVar("_T")
def _list_implementations(which: Type[_T]) -> Set[Type[_T]]:
"""
1. Get the fully-qualified name of the class
2. See if the necessary inputs are declared in any of our config files to construct a concrete implementation of it
3. Return a list of the concrete subclasses that are available in the config files so we can determine priority and construct it
My caller may indeed want to pass an abstract class which they require an implementation of.
"""
pass
def get_implementation(which: Type[_T]) -> _T:
available_impls = _list_implementations(which)
# get the priority of each class from a classmethod
# load the one with the highest priority
return highest_priority_class
Is mypy open to adjusting its interpretation here, or perhaps providing another type annotation which would accept abstract classes?
@Rahix Agreed, thanks for the pointer! The people there are having exactly the same issue I am.
I'm leaning towards allowing abstract classes to be used without restriction.
Type[x]is already unsafe since__init__can be overriden with a different signature, and I expect that the extra unsafety from allowing abstract classes would be marginal in practice. We just need to document the potential issues clearly.
@JukkaL
Do you still think this? Should disabling this be tossed behind a flag, or just completely be deleted?
