mypy

mypy copied to clipboard

First of all, I'm using Mypy 0.770.

I'd like to annotate a function in the following way:

T = TypeVar('T')

class JSONSerialized(str, Generic[T]):
    """Strong type for the JSON serialized version of a type."""

@overload
def json_serialize(value: None) -> None:
    """Serialize value to its strong-typed JSON string type."""

@overload
def json_serialize(value: T) -> JSONSerialized[T]:
    """Serialize value to its strong-typed JSON string type."""

def json_serialize(value: Optional[T]) -> Optional[JSONSerialized[T]]:
    """Serialize value to its strong-typed JSON string type."""
    if value is None:
        return None

    return cast(JSONSerialized[T], json.dumps(to_json(value)))

The idea here is that I know that a string has been serialized from a given type (converted to a json-compatible type by the to_json function which is not depicted here.

The overloads don't work, however. I'd like them to inform mypy that this only returns None if None goes in, otherwise it doesn't. Unfortunately, since T is an unconstrained TypeVar, it also matches None, and mypy gives me the following error:

error: Overloaded function signatures 1 and 2 overlap with incompatible return types

Thus, I need a way to say "anything but None" (as a TypeVar in this case), but I can't seem to find any way to do so, so I believe this may end up being a feature request. I've seen similar behaviour being asked about in StackOverflow:

https://stackoverflow.com/questions/57854871/exclude-type-in-python-typing-annotation https://stackoverflow.com/questions/47215682/what-type-represents-typing-any-except-none?noredirect=1&lq=1 https://stackoverflow.com/questions/58612057/how-to-make-a-generic-typevar-in-mypy-be-non-optional?noredirect=1&lq=1

Also, I'm not using any flags that change mypy's behaviour about None.

+1. I think introducing a NotNone type is the way to go here.

In terms of implementation, I'd imagine NotNone would become the primary base type (except for None), and object would be treated as Union[NotNone, None] (aka Optional[NotNone]). Note that this is also how Kotlin is designed, where Any is the superclass of everything except Null, and the true top-type is Any? (aka Any | Null).

If that existed, the example above could be rewritten like this:

TNotNone = TypeVar('TNotNone', bound=NotNone)
T = TypeVar('T')

class JSONSerialized(str, Generic[TNotNone]):
    """Strong type for the JSON serialized version of a type."""

@overload
def json_serialize(value: None) -> None:
    """Serialize value to its strong-typed JSON string type."""

@overload
def json_serialize(value: TNotNone) -> JSONSerialized[TNotNone]:
    """Serialize value to its strong-typed JSON string type."""

@overload
def json_serialize(value: Optional[T]) -> Optional[JSONSerialized[T]]:
    """Serialize value to its strong-typed JSON string type."""

def json_serialize(value: Optional[T]) -> Optional[JSONSerialized[T]]:
    """Serialize value to its strong-typed JSON string type."""
    if value is None:
        return None

    return cast(JSONSerialized[T], json.dumps(to_json(value)))

(The third overload is needed for calling with a value of type object, which doesn't match None or TNotNone. In that case, T would be inferred as NotNone.)

@russelldavis I don't think NotNone solves the general problem, which seems to be constraining a TypeVar that's bound to a Union.

Would have to be something along the lines of

class A:
    pass

class B(A):
    pass

class C(A):
    pass

class D(A):
    pass

class E(A):
    pass

class F(E):
    pass

_TANotDOrE = TypeVar("_TANotDOrE", bound=A, exclude=(D, E))

Where _TANotDOrE is essentially equivalent to TypeVar("_TANotDOrE", A, B, C).

Has the best solution been invented for this case? I have just run into the same 'unconstrained TypeVar contains None' pitfall.

Should probably link the addition of Not as the ideal solution: https://github.com/python/typing/issues/801