mypy
mypy copied to clipboard
python
int is not a Number?
n : Number = 5
produces
test_compiler.py:18: error: Incompatible types in assignment (expression has type "int", variable has type "Number")
Which it probably shouldn't because isinstance(n, Number) == True
Thank you for reporting this!
It looks like this is mostly a typeshed issue. Structursl subtyping (see PR #3132) will help, but it looks like still some changes will be necessary in typeshed, since we can't declare Number a protocol (it is too trivial currently).
@JelleZijlstra what do you think?
Not sure we need to keep this open, if it can be fixed in typeshed once protocols land. (Maybe there should be an omnibus issue for things we can change once it lands?)
Also see PEP 484, which has this to say:
PEP 3141 defines Python's numeric tower, and the stdlib module numbers implements the corresponding ABCs ( Number , Complex , Real , Rational and Integral ). There are some issues with these ABCs, but the built-in concrete numeric classes complex , float and int are ubiquitous (especially the latter two :-).
Rather than requiring that users write import numbers and then use numbers.Float etc., this PEP proposes a straightforward shortcut that is almost as effective: when an argument is annotated as having type float , an argument of type int is acceptable; similar, for an argument annotated as having type complex , arguments of type float or int are acceptable. This does not handle classes implementing the corresponding ABCs or the fractions.Fraction class, but we believe those use cases are exceedingly rare.
I am not too familiar with the numeric tower, but maybe we should just go with PEP 484's recommendation and avoid the numbers ABCs.
numbers.Number in particular doesn't seem very useful since it declares no members, so logically mypy shouldn't allow you to do anything with a variable declared as a Number. It looks like the others can be made into Protocols easily when that lands.
What are concrete use cases where one would use the numbers ABCs instead of int/float/complex? Maybe numpy numeric types?
What are concrete use cases where one would use the numbers ABCs instead of int/float/complex? Maybe numpy numeric types?
It's come up a few times before, the only places I found that were using it were just trying to be hyper-correct. E.g. numpy supports it but barely mentions it in its docs.
[Warning: Luddite mode on] IMO PEP 3141 was a flight of fancy and has never produced much useful effect. Pythoneers are generally to close to practice to care about the issue, most of the time the code just works through the magic of duck typing, and when it doesn't people typically say "well that's too fancy anyways". :-)
I'm not sure if the other ABCs beyond Number are very straightforward either. It would be an interesting exercise for somebody to write a prototype stub (using the protocol PR) that only supports one operation (such as addition) for Complex, Real and friends. It would have to work with int, float and user-defined classes -- and probably also with mixtures of those when it makes sense.
However, I don't believe that the numeric tower is going to be very useful for type checking. The current types seem to be good enough for the vast majority of use cases, and just providing another set of types for essentially the same purpose seems like unnecessary complexity. In my opinion, a much more promising (and somewhat related) project would be figuring out how to type check code using numpy.
In my opinion, a much more promising (and somewhat related) project would be figuring out how to type check code using numpy.
I agree this looks very interesting. This would require some simple dependent types, since fixed size arrays/matrices are everywhere in mypy. The rules for those are non-trivial, e.g. matrix[n, m] * matrix[m, k] == matrix[n, k]. But I think fixed size arrays will be quite useful on its own even outside numpy.
It would be really cool at least to document it a bit more clearly than the PEP 484 quote, which I read as
"You don't need to use the numbers.* classes but you can"
when it appears that the real situation is
"The numbers.* classes do not work with typing".
I (quite reasonably?) thought that isinstance(x, cls) implies that you can have the
x : cls
type declaration.
Are there other places where this is not the case?
Just noting that I seem to have a use-case affected by this; I'm adding annotations for use w/ mypy to some code that takes latitude/longitude values; so,
def foo(lat, lng):
# type: (?, ?) -> None
I started with float, but passing an integer is okay. That led me to Union[float, int], and subsequently #2128; however, Decimal is also okay — ideally, I just work in whatever types get input to the function and let duck typing do its thing, but Union[float, int, Decimal…] is not what I really want to say. So, numbers.Real, and that brings one to this bug.
(I agree with tjltjl's reasoning that if the specification is x: Cls, then passing a foo, where isinstance(foo, Cls), should work.)
numbers.Real doesn't require any special actions, it will be fixed automatically when protocols PR lands (unlike numbers.Number that also might require some changes in typeshed).
@ilevkivskyi, has that PR landed? Because I have just encountered the same problem with Mypy 0.590:
def bug(n: Real = 1) -> Real:
return n + 1
# error: Incompatible default for argument "n" (default has type "int", argument has type "Real")
Motivating example is HypothesisWorks/hypothesis#200; we have several test strategies for some type of number within a range, and the bounding values can be specified as int/float/fraction - so it would be really nice to show Real instead of that union!
Yes, but those types are still not labeled as protocols. I personally don't see any harm in labelling them as protocols. Note that the other type is Any for all methods, but I don't think this will introduce any additional unsafety as compared to current situation. The problem however is that Number can't be made a protocol (because it is empty), but it is a base class for all other numbers and protocols can't inherit from non-protocols, so at least for now they stay as ABCs, i.e. this will need a .register() support, see https://github.com/python/mypy/issues/2922
One consideration around the numeric tower is that when a function returns numpy.float, it would be nice to mark it as returning numbers.Real, since
(1) I don't want to force people to fix type annotations because they happened to return a float in the future (2) You could see both numpy.float and python float returned from the same function for convenience.
One possible option is to add a bit of lie to typeshed, see https://github.com/python/typeshed/pull/3108.
Another option is to reject the use of Number etc. as types (and maybe give a link to documentation that explains why this isn't supported and what to do instead).
Another option is to reject the use of
Numberetc. as types (and maybe give a link to documentation that explains why this isn't supported and what to do instead).
This would be a bit sad, especially taking into account this is a 12-th most liked issue on the tracker. TBH I forgot the details of the discussion, but if the typeshed attempt will work out are there any reasons not to do this?
but if the typeshed attempt will work out are there any reasons not to do this?
I'm wondering if the types will be useful for their intended purpose. If not, the effort may be wasted.
Doesn't seem like this will ever get fixed. Can you at least add a note to the user doc somewhere?
Personally I'm not fussed that (mypy thinks) int is not a Number, because I don't have any use for Number. I am inconvenienced by the fact that (mypy thinks) float and int are not Real, though, since I now have a case where I need to use Union[Real, float] to satisfy mypy.
Doesn't seem like this will ever get fixed. Can you at least add a note to the user doc somewhere?
Perhaps somebody would like to contribute an item about this to the common issues section in the documentation?
Is it possible, one day, to make a MyPy plugin that can answer the question: "is float a subtype of Real?" Is it possible for plugins to be aware of ABC subtype registration?
This pattern of registering subtypes of ABCs comes up every now and then. In JAX, they have a similar registration mechanism. It would be amazing to have a JAX PyTree type that understood all of the subtypes that are registered.
For now, I have to choose between using an alias to All and using casts.
I assume that this is also the same root issue:
import numbers
from typing import Union
FixedRational = Union[int, numbers.Rational]
def broken(n: numbers.Rational) -> int:
return n.numerator * n.denominator
def fixed(n: FixedRational) -> int:
return n.numerator * n.denominator
print(broken(3))
print(fixed(3))
Which produces: error: Argument 1 to "broken" has incompatible type "int"; expected "Rational" [arg-type]
I was trying to create a modified Fraction class customized to my needs, and the numbers module is fairly important to that effort. I suppose I can manually create the required Unions, but I'm not sure I understand why this is hard to fix...
Stumbled into this right now, annotating a dataclass that can perfectly hold a float or a Fraction: a musical interval which can be both: a rational when we’re lucky—and then it should show to the user as a fraction—and a float when we’re not as lucky, as with almost every regular temperament. Both cases arise more or less equally often.
I suppose I can alias Union[int, Fraction, float] for a while but it would be really nice if this gets resolved in some manner. :sun_with_face:
This could be solved in typeshed. Right now, types like int and float don't inherit from numbers.Integral and numbers.Real -- I don't recall if there's a technicality involved or whether it's just work (or possibly fear of slowing down everything). At runtime we "register" these virtual subclassing relationships, but static type checkers don't handle ABC.register().
PS. PEP 484 and mypy do treat int as a virtual subclass of float, so you don't need Union[int, float].
The technicality I remember being discussed was https://github.com/python/typeshed/issues/3195 We could also make numbers.Real and numbers.Integral (but not numbers.Number) protocols.
Hm, would it be so terrible if numbers.Number in typeshed didn't have __floor__ and __ceil__? Or to pretend float has them? Also, probably few would care if we made Number a protocol with the same methods as Complex -- or at least the intersection of Complex and Decimal -- it seems Decimal is the one numeric type that inherits from Number but not from Complex.
PS. PEP 484 and mypy do treat int as a virtual subclass of float, so you don't need
Union[int, float].
(Oh. Right, thanks! I even read about this thing in this same issue yesterday before posting, but for some reason forgot that entirely when started writing. :thinking:)