Infer descriptor type

[iwanb]

This is a copy of this enhancement I proposed for mypy: https://github.com/python/mypy/issues/10246

I use this inference in a project through a mypy plugin, I'm trying out pyrefly so I'm curious if this is something you'd consider allowing or not (feel free to close the issue in that case).

Feature

For the common case of a descriptor which behaves like a normal attribute, infer:

class X:
    y: int = descriptor()

to:

class X:
    y: descriptor[int] = descriptor()

if descriptor is a Generic with one parameter. Extra checks should also be done to make sure that the descriptor is properly typed to behave like an attribute of that type (__get__ and __set__ expect that type on an instance).

Pitch

Type annotations apply to object instances by default, so it's strange to see y: descriptor[int] as type annotation while x.y returns an int. You could also see a normal attribute as an invisible descriptor whose behavior is to set dict[name] (and raise AttributeError on the class), so this could be a way to unify these things.

Other

This is kind of how dataclasses are typed (although dataclasses have more magic involved).

Possibly this could be opt-in through some mechanism like a metaclass or a class decorator, to make it more explicit.

sandbox example

[stroxler]

Thanks for the suggestion! This seems like a great feature to have

If we were a stand-alone project I'd have no hesitation (other than we might not get to it for a bit), as it is I think we just need to make sure we don't diverge from other type checkers.

I think that it should be okay, since the presence of an explicit annotation means this reduces to a type inference question (it's almost a form of contextual typing) but it would be good to get another opinion, and also keep an eye on the mypy issue

@rchen152 any thoughts?

[stroxler]

Claiming this for now since I think it needs some discussion, and it's in a part of the codebase I'm looking at anyway

[rchen152]

This is an interesting idea! A couple thoughts:

• As Steven mentioned, diverging from other type checkers is a risk. If we support this, I could imagine a library that uses pyrefly for type checking shipping with type annotations like:

class X:
    y: int = descriptor()

and not realizing that other type checkers will get a different type for X.y.

• How would this work in type stubs? The typical way of declaring attribute types in stubs:

class X:
    y: int

wouldn't work because the information that y is a descriptor would be lost. In general, the fact that looking at the type annotation isn't enough to determine the type seems like it could be a little tricky to deal with.

[iwanb]

I agree that it would be better as a fleshed out PEP :)

The idea is that the descriptor returns the type you use to annotate on instances of the object. So for a type stub, you can ignore descriptors completely. For typing purposes obj.attr returns the same type whether it's an attribute or a descriptor. Knowing the descriptor is only useful if it returns something else when used on the class object.

I've actually used this pattern with properties, a property behaves like an attribute, so you can have a stub with the type attribute and the subclass with a property. IMO that should type check (but I understand you can argue it should not):

class A:
   myattr: int

class B(A):
  @property
  def myattr(self) -> int:
    return 1

And properties are descriptors.

[stroxler]

Interesting - I would actually argue that

class A:
   myattr: int

class B(A):
  @property
  def myattr(self) -> int:
    return 1

should not type check, at least under the strictest setting (we might be able to use a dedicated error code for the error so it's easy to bypass) because A().myattr = 15 is legal and B().myattr = 15 will crash the program.

But I agree that if a descriptor has both a getter and a setter, and they are simple enough to analyze as type-safe with respect to the annotation, then we probably can get away with this feature.

[rchen152]

So for a type stub, you can ignore descriptors completely.

I don't understand this statement. Using the sandbox example above:

class ExampleWish:
    x: int = Constant(10)
reveal_type(ExampleWish.x)  # int, would like Constant[int]
reveal_type(ExampleWish().x)  # int

how would you write a stub definition for ExampleWish so that a type checker reading only the stub, without access to the source code, would type ExampleWish.x as Constant[int]?

To be clear, I think this is a solvable problem. There's precedent for, e.g., defining enum members in a special way. Just wanted to flag it as something I think we'll need to address.

[iwanb]

how would you write a stub definition for ExampleWish so that a type checker reading only the stub, without access to the source code, would type ExampleWish.x as Constant[int]?

You would not, the type checker would refuse ExampleWish.x, unless you have a subclass which has a concrete implementation, or you type it using the descriptor. So "you can ignore descriptors completely" was indeed an overstatement, but you can ignore it as long as you only work with instance attributes (which I think is the majority of code).

should not type check, at least under the strictest setting (we might be able to use a dedicated error code for the error so it's easy to bypass) because A().myattr = 15 is legal and B().myattr = 15 will crash the program.

You're right I did not take into account immutability in my example, so ideally the descriptor should have both the getter and setter (and deleter in theory?).

Come to think of it, is there even a way to mark an attribute as immutable in Python? I only saw typing.ReadOnly which is specific to TypedDict.

EDIT: ah, I suppose it's Final? Although it prevents reassigning in the subclass, which is not what I want here. https://docs.python.org/3/library/typing.html#typing.Final

[github-actions[bot]]

This issue has someone assigned, but has not had recent activity for more than 2 weeks.

If you are still working on this issue, please add a comment so everyone knows. Otherwise, please unassign yourself and allow someone else to take over.

Thank you for your contributions!