[red-knot] Literal special form

[Glyphack]

Handling Literal type in annotations.

Resolves: #13672

Implementation

Since Literals are not a fully defined type in typeshed. I used a trick to figure out when a special form is a literal. When we are inferring assignment types I am checking if the type of that assignment was resolved to typing.SpecialForm and the name of the target is Literal if that is the case then I am re creating a new instance type and set the known instance field to KnownInstance:Literal.

Why not defining a new type?

From this issue I learned that we want to resolve members to SpecialMethod class. So if we create a new instance here we can rely on the member resolving in that already exists.

Tests

https://typing.readthedocs.io/en/latest/spec/literal.html#equivalence-of-two-literals Since the type of the value inside Literal is evaluated as a Literal(LiteralString, LiteralInt, ...) then the equality is only true when types and value are equal.

https://typing.readthedocs.io/en/latest/spec/literal.html#legal-and-illegal-parameterizations

The illegal parameterizations are mostly implemented I'm currently checking the slice expression and the slice type to make sure it's valid.

Not covered:

1. I did not find an easy way to error on things like Literal["foo".replace("o", "b")] because I cannot fully disable attribute expressions in the Literal since enum members are allowed.
2. parenthesized Tuples are not allowed. Although pyright allows this in the doc is stated that tuples containing valid literal types are illegal. Tuples are valid in case of Literal["w", "r"] for example.

The union creation with Literals is not working because I saw comments about Union not implemented yet. https://typing.readthedocs.io/en/latest/spec/literal.html#shortening-unions-of-literals

Summary

Test Plan

[github-actions[bot]]

ruff-ecosystem results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

✅ ecosystem check detected no linter changes.

[Glyphack]

I applied the comments will spend another day on adding diagnostic messages for https://typing.readthedocs.io/en/latest/spec/literal.html#legal-and-illegal-parameterizations

[Glyphack]

Okay I added more parts of the legal and illegal parameters from the spec. Right now we have:

1. All Literal types I managed to do this through a lot of recursion.
2. Nested Literals

I think the remaining part is assignability check. Right now the Literals are unwrapped to their inner type I don't think this is the right way, is it? It works in the tests but I'm not sure if Literal types should carry some special flags with themselves.

I did not find an easy way to error on things like Literal["foo".replace("o", "b")] because I cannot fully disable attribute expressions in the Literal since enum members are allowed. I can do the same I did on nested literals:

1. Check if it's attribute
2. Check the type of value and if it's a class and has Enum in bases otherwise emit diagnostic.

Does this sound good?

parenthesized Tuples are not allowed. Although pyright allows this in the doc is stated that tuples containing valid literal types are illegal. Tuples are valid in case of Literal["w", "r"] for example.

Also I'm not correctly joining union when it's possible. I left it as a todo in the tests:

    # TODO: revealed: Literal[1, 2, 3, "foo", 5] | None
    reveal_type(union_var)  # revealed: Literal[1, 2, 3] | Literal["foo"] | Literal[5] | None

Please let me know what do you think.

[carljm]

Explaining the commit I just pushed:

My intent in suggesting InstanceType was that it would not be a Salsa interned struct, but just a regular struct that would be stored inline in Type, so we wouldn't add an extra layer of database queries. That's why I was discussing the size of Type. I wanted to verify that this really worked, and didn't increase the size of Type, before making the suggestion -- and once I had verified that, I figured I might as well push the changes and not ask you to make them again :)

[Glyphack]

Thanks for your time and guidance, I appreciate it.