Experiment: Let `SupportsDunderLT` return `object`.

[randolf-scholz]

Fixes #12562

[github-actions[bot]]

Diff from mypy_primer, showing the effect of this PR on open source code:

jax (https://github.com/google/jax)
+ jax/_src/pallas/mosaic/pipeline.py:913: error: Unused "type: ignore" comment  [unused-ignore]

ibis (https://github.com/ibis-project/ibis)
- ibis/expr/types/relations.py:3028: error: Value of type variable "SupportsRichComparisonT" of "sorted" cannot be "NumericValue | float"  [type-var]

[github-actions[bot]]

Diff from mypy_primer, showing the effect of this PR on open source code:

rich (https://github.com/Textualize/rich)
- rich/syntax.py:491: error: "lexer" has type "Lexer" which does not implement __bool__ or __len__ so it could always be true in boolean context  [truthy-bool]

aiohttp (https://github.com/aio-libs/aiohttp)
+ aiohttp/client_reqrep.py:1052: error: Unused "type: ignore" comment  [unused-ignore]

pwndbg (https://github.com/pwndbg/pwndbg)
+ pwndbg/gdblib/onegadget.py: note: In function "check_constraint":
+ pwndbg/gdblib/onegadget.py:484: error: Argument 1 to "find" has incompatible type "CheckSatResult"; expected "int | Value | None"  [arg-type]
+ pwndbg/gdblib/onegadget.py:496: error: Argument 1 to "find" has incompatible type "CheckSatResult"; expected "int | Value | None"  [arg-type]

pandas-stubs (https://github.com/pandas-dev/pandas-stubs)
+ pandas-stubs/_libs/missing.pyi:6: error: Return type "None" of "__bool__" incompatible with return type "bool" in supertype "object"  [override]
+ pandas-stubs/core/indexes/base.pyi:341: error: Incompatible types in assignment (expression has type "EllipsisType", base class "object" defined the type as "Callable[[object], bool]")  [assignment]

spark (https://github.com/apache/spark)
+ python/pyspark/sql/column.py:1529: error: Return type "None" of "__bool__" incompatible with return type "bool" in supertype "object"  [override]

werkzeug (https://github.com/pallets/werkzeug)
+ src/werkzeug/local.py:555: error: Incompatible types in assignment (expression has type "_ProxyLookup", base class "object" defined the type as "Callable[[object], bool]")  [assignment]
+ src/werkzeug/local.py:555: note: "_ProxyLookup.__call__" has type "Callable[[Arg(LocalProxy[Any], 'instance'), VarArg(Any), KwArg(Any)], Any]"

jax (https://github.com/google/jax)
+ jax/_src/pallas/mosaic/pipeline.py:913: error: Unused "type: ignore" comment  [unused-ignore]

core (https://github.com/home-assistant/core)
+ homeassistant/helpers/entity_platform.py:621: error: Unused "type: ignore" comment  [unused-ignore]

ibis (https://github.com/ibis-project/ibis)
- ibis/expr/types/relations.py:3028: error: Value of type variable "SupportsRichComparisonT" of "sorted" cannot be "NumericValue | float"  [type-var]

[github-actions[bot]]

Diff from mypy_primer, showing the effect of this PR on open source code:

jax (https://github.com/google/jax)
+ jax/_src/pallas/mosaic/pipeline.py:913: error: Unused "type: ignore" comment  [unused-ignore]

ibis (https://github.com/ibis-project/ibis)
- ibis/expr/types/relations.py:3028: error: Value of type variable "SupportsRichComparisonT" of "sorted" cannot be "NumericValue | float"  [type-var]

[srittau]

I'm not a fan of this change. This would decrease the type safety for the common case. For example, TestCase.assertGreater really wants an object that returns a bool and not just any overloaded object.

[JelleZijlstra]

The change here would affect objects that do have an __lt__ or similar method, but where that method doesn't return a bool. I don't think that's likely to cause a lot of real unsafety.

[github-actions[bot]]

Diff from mypy_primer, showing the effect of this PR on open source code:

ibis (https://github.com/ibis-project/ibis)
- ibis/expr/types/relations.py:2916: error: Value of type variable "SupportsRichComparisonT" of "sorted" cannot be "NumericValue | float"  [type-var]

jax (https://github.com/google/jax)
+ jax/_src/pallas/mosaic/pipeline.py:950: error: Unused "type: ignore" comment  [unused-ignore]

[srittau]

I'm still opposed to this. Overloading the comparison operators to return something other than bool is unexpected and using # type: ignore to make this explicit is acceptable.

[jorenham]

I'm still opposed to this. Overloading the comparison operators to return something other than bool is unexpected and using # type: ignore to make this explicit is acceptable.

I'd agree with you if it indeed would be fixeable with one or two # type: ignore comments. But as I've shown in https://github.com/python/typeshed/issues/12562, this unfortunately isn't the case for user-defined types that return something other than a bool from e.g. __lt__, which currently is rejected when used with e.g. builtins.min.

[randolf-scholz]

I can't help but be reminded of this comment https://github.com/python/mypy/issues/1020#issuecomment-1318657466.

As someone who uses numpy daily, I don't find it surprising that < or > might return something other than a bool. If one finds that surprising or a code smell, using a linter might be more appropriate. Imo, the stubs should reflect runtime behavior as best as possible.

[randolf-scholz]

@srittau @JelleZijlstra With this PR seemingly being stuck, can we at least consider revisiting the SupportsBool approach #12939 ? If this weaker modification is agreeable to @srittau, maybe we can merge it and reconsider this PR at a later point.

[github-actions[bot]]

Diff from mypy_primer, showing the effect of this PR on open source code:

ibis (https://github.com/ibis-project/ibis)
- ibis/expr/types/relations.py:2914: error: Value of type variable "SupportsRichComparisonT" of "sorted" cannot be "NumericValue | float"  [type-var]

jax (https://github.com/google/jax)
+ jax/_src/pallas/mosaic/pipeline.py:981: error: Unused "type: ignore" comment  [unused-ignore]

[github-actions[bot]]

Diff from mypy_primer, showing the effect of this PR on open source code:

ibis (https://github.com/ibis-project/ibis)
- ibis/expr/types/relations.py:2924: error: Value of type variable "SupportsRichComparisonT" of "sorted" cannot be "NumericValue | float"  [type-var]

jax (https://github.com/google/jax)
+ jax/_src/pallas/mosaic/pipeline.py:981: error: Unused "type: ignore" comment  [unused-ignore]

[github-actions[bot]]

Diff from mypy_primer, showing the effect of this PR on open source code:

ibis (https://github.com/ibis-project/ibis)
- ibis/expr/types/relations.py:2946: error: Value of type variable "SupportsRichComparisonT" of "sorted" cannot be "NumericValue | float"  [type-var]

scipy (https://github.com/scipy/scipy)
- scipy/spatial/tests/test_spherical_voronoi.py:194: error: Value of type variable "SupportsRichComparisonT" of "sorted" cannot be "floating[Any]"  [type-var]

rclip (https://github.com/yurijmikhalevich/rclip)
- rclip/model.py:212: error: Argument "key" to "sorted" has incompatible type "Callable[[tuple[floating[Any], int]], floating[Any]]"; expected "Callable[[tuple[floating[Any], int]], SupportsDunderLT[Any] | SupportsDunderGT[Any]]"  [arg-type]
- rclip/model.py:212: error: Incompatible return value type (got "floating[Any]", expected "SupportsDunderLT[Any] | SupportsDunderGT[Any]")  [return-value]

jax (https://github.com/google/jax)
+ jax/_src/pallas/mosaic/pipeline.py:982: error: Unused "type: ignore" comment  [unused-ignore]

[github-actions[bot]]

Diff from mypy_primer, showing the effect of this PR on open source code:

ibis (https://github.com/ibis-project/ibis)
- ibis/expr/types/relations.py:2946: error: Value of type variable "SupportsRichComparisonT" of "sorted" cannot be "NumericValue | float"  [type-var]

scipy (https://github.com/scipy/scipy)
- scipy/spatial/tests/test_spherical_voronoi.py:194: error: Value of type variable "SupportsRichComparisonT" of "sorted" cannot be "floating[Any]"  [type-var]

jax (https://github.com/google/jax)
+ jax/_src/pallas/mosaic/pipeline.py:982: error: Unused "type: ignore" comment  [unused-ignore]

rclip (https://github.com/yurijmikhalevich/rclip)
- rclip/model.py:212: error: Argument "key" to "sorted" has incompatible type "Callable[[tuple[float64, int]], float64]"; expected "Callable[[tuple[float64, int]], SupportsDunderLT[Any] | SupportsDunderGT[Any]]"  [arg-type]
- rclip/model.py:212: error: Incompatible return value type (got "float64", expected "SupportsDunderLT[Any] | SupportsDunderGT[Any]")  [return-value]

[github-actions[bot]]

Diff from mypy_primer, showing the effect of this PR on open source code:

colour (https://github.com/colour-science/colour)
- colour/algebra/interpolation.py:692: error: Value of type variable "SupportsRichComparisonT" of "min" cannot be "floating[_16Bit] | floating[_32Bit] | float64"  [type-var]
- colour/algebra/interpolation.py:693: error: Value of type variable "SupportsRichComparisonT" of "max" cannot be "floating[_16Bit] | floating[_32Bit] | float64"  [type-var]
- colour/algebra/interpolation.py:700: error: Value of type variable "SupportsRichComparisonT" of "min" cannot be "floating[_16Bit] | floating[_32Bit] | float64"  [type-var]
- colour/colorimetry/spectrum.py:1180: error: Value of type variable "SupportsRichComparisonT" of "max" cannot be "int | float | floating[_16Bit] | floating[_32Bit]"  [type-var]
- colour/colorimetry/spectrum.py:1181: error: Value of type variable "SupportsRichComparisonT" of "min" cannot be "int | float | floating[_16Bit] | floating[_32Bit]"  [type-var]
- colour/colorimetry/spectrum.py:1611: error: Value of type variable "SupportsRichComparisonT" of "max" cannot be "floating[_16Bit] | floating[_32Bit] | float64"  [type-var]
- colour/plotting/volume.py:261: error: Value of type variable "SupportsRichComparisonT" of "sorted" cannot be "floating[_16Bit] | floating[_32Bit] | float64"  [type-var]
- colour/plotting/colorimetry.py:195: error: No overload variant of "max" matches argument types "floating[_16Bit] | floating[_32Bit] | float64", "floating[_16Bit] | floating[_32Bit] | float64"  [call-overload]
- colour/plotting/colorimetry.py:195: note: Possible overload variants:
- colour/plotting/colorimetry.py:195: note:     def [SupportsRichComparisonT: SupportsDunderLT[Any] | SupportsDunderGT[Any]] max(SupportsRichComparisonT, SupportsRichComparisonT, /, *_args: SupportsRichComparisonT, key: None = ...) -> SupportsRichComparisonT
- colour/plotting/colorimetry.py:195: note:     def [_T] max(_T, _T, /, *_args: _T, key: Callable[[_T], SupportsDunderLT[Any] | SupportsDunderGT[Any]]) -> _T
- colour/plotting/colorimetry.py:195: note:     def [SupportsRichComparisonT: SupportsDunderLT[Any] | SupportsDunderGT[Any]] max(Iterable[SupportsRichComparisonT], /, *, key: None = ...) -> SupportsRichComparisonT
- colour/plotting/colorimetry.py:195: note:     def [_T] max(Iterable[_T], /, *, key: Callable[[_T], SupportsDunderLT[Any] | SupportsDunderGT[Any]]) -> _T
- colour/plotting/colorimetry.py:195: note:     def [SupportsRichComparisonT: SupportsDunderLT[Any] | SupportsDunderGT[Any], _T] max(Iterable[SupportsRichComparisonT], /, *, key: None = ..., default: _T) -> SupportsRichComparisonT | _T
- colour/plotting/colorimetry.py:195: note:     def [_T1, _T2] max(Iterable[_T1], /, *, key: Callable[[_T1], SupportsDunderLT[Any] | SupportsDunderGT[Any]], default: _T2) -> _T1 | _T2
- colour/plotting/colorimetry.py:195: error: Value of type variable "SupportsRichComparisonT" of "min" cannot be "floating[_16Bit] | floating[_32Bit] | float64"  [type-var]
- colour/plotting/colorimetry.py:196: error: No overload variant of "min" matches argument types "floating[_16Bit] | floating[_32Bit] | float64", "floating[_16Bit] | floating[_32Bit] | float64"  [call-overload]
- colour/plotting/colorimetry.py:196: note: Possible overload variants:
- colour/plotting/colorimetry.py:196: note:     def [SupportsRichComparisonT: SupportsDunderLT[Any] | SupportsDunderGT[Any]] min(SupportsRichComparisonT, SupportsRichComparisonT, /, *_args: SupportsRichComparisonT, key: None = ...) -> SupportsRichComparisonT
- colour/plotting/colorimetry.py:196: note:     def [_T] min(_T, _T, /, *_args: _T, key: Callable[[_T], SupportsDunderLT[Any] | SupportsDunderGT[Any]]) -> _T
- colour/plotting/colorimetry.py:196: note:     def [SupportsRichComparisonT: SupportsDunderLT[Any] | SupportsDunderGT[Any]] min(Iterable[SupportsRichComparisonT], /, *, key: None = ...) -> SupportsRichComparisonT
- colour/plotting/colorimetry.py:196: note:     def [_T] min(Iterable[_T], /, *, key: Callable[[_T], SupportsDunderLT[Any] | SupportsDunderGT[Any]]) -> _T
- colour/plotting/colorimetry.py:196: note:     def [SupportsRichComparisonT: SupportsDunderLT[Any] | SupportsDunderGT[Any], _T] min(Iterable[SupportsRichComparisonT], /, *, key: None = ..., default: _T) -> SupportsRichComparisonT | _T
- colour/plotting/colorimetry.py:196: note:     def [_T1, _T2] min(Iterable[_T1], /, *, key: Callable[[_T1], SupportsDunderLT[Any] | SupportsDunderGT[Any]], default: _T2) -> _T1 | _T2
- colour/plotting/colorimetry.py:196: error: Value of type variable "SupportsRichComparisonT" of "max" cannot be "floating[_16Bit] | floating[_32Bit] | float64"  [type-var]
- colour/plotting/colorimetry.py:223: error: Value of type variable "SupportsRichComparisonT" of "min" cannot be "floating[_16Bit] | floating[_32Bit] | float64"  [type-var]
- colour/plotting/colorimetry.py:223: error: Value of type variable "SupportsRichComparisonT" of "max" cannot be "floating[_16Bit] | floating[_32Bit] | float64"  [type-var]
- colour/plotting/colorimetry.py:224: error: Value of type variable "SupportsRichComparisonT" of "max" cannot be "floating[_16Bit] | floating[_32Bit] | float64"  [type-var]
- colour/plotting/colorimetry.py:229: error: List item 0 has incompatible type "tuple[floating[_16Bit] | floating[_32Bit] | float64, int]"; expected "_SupportsArray[dtype[Any]] | _NestedSequence[_SupportsArray[dtype[Any]]]"  [list-item]
- colour/plotting/colorimetry.py:231: error: List item 2 has incompatible type "tuple[floating[_16Bit] | floating[_32Bit] | float64, int]"; expected "_SupportsArray[dtype[Any]] | _NestedSequence[_SupportsArray[dtype[Any]]]"  [list-item]
- colour/plotting/colorimetry.py:243: error: Value of type variable "SupportsRichComparisonT" of "max" cannot be "floating[_16Bit] | floating[_32Bit] | float64"  [type-var]
- colour/plotting/colorimetry.py:406: error: Value of type variable "SupportsRichComparisonT" of "min" cannot be "floating[_16Bit] | floating[_32Bit] | float64"  [type-var]
- colour/plotting/colorimetry.py:407: error: Value of type variable "SupportsRichComparisonT" of "max" cannot be "floating[_16Bit] | floating[_32Bit] | float64"  [type-var]
- colour/plotting/colorimetry.py:423: error: Value of type variable "SupportsRichComparisonT" of "min" cannot be "floating[_16Bit] | floating[_32Bit] | float64"  [type-var]
- colour/plotting/colorimetry.py:424: error: Value of type variable "SupportsRichComparisonT" of "max" cannot be "floating[_16Bit] | floating[_32Bit] | float64"  [type-var]
- colour/plotting/colorimetry.py:565: error: Value of type variable "SupportsRichComparisonT" of "min" cannot be "floating[_16Bit] | floating[_32Bit] | float64"  [type-var]
- colour/plotting/colorimetry.py:566: error: Value of type variable "SupportsRichComparisonT" of "max" cannot be "floating[_16Bit] | floating[_32Bit] | float64"  [type-var]
- colour/plotting/colorimetry.py:768: error: Value of type variable "SupportsRichComparisonT" of "min" cannot be "floating[_16Bit] | floating[_32Bit] | float64"  [type-var]
- colour/plotting/colorimetry.py:768: error: Value of type variable "SupportsRichComparisonT" of "max" cannot be "floating[_16Bit] | floating[_32Bit] | float64"  [type-var]
- colour/examples/algebra/examples_interpolation.py:142: error: Value of type variable "SupportsRichComparisonT" of "min" cannot be "floating[_16Bit] | floating[_32Bit] | float64"  [type-var]
- colour/examples/algebra/examples_interpolation.py:143: error: Value of type variable "SupportsRichComparisonT" of "max" cannot be "floating[_16Bit] | floating[_32Bit] | float64"  [type-var]
- colour/examples/algebra/examples_interpolation.py:155: error: Value of type variable "SupportsRichComparisonT" of "min" cannot be "floating[_16Bit] | floating[_32Bit] | float64"  [type-var]
- colour/examples/algebra/examples_interpolation.py:156: error: Value of type variable "SupportsRichComparisonT" of "max" cannot be "floating[_16Bit] | floating[_32Bit] | float64"  [type-var]

jax (https://github.com/google/jax)
+ jax/_src/pallas/mosaic/pipeline.py:1233: error: Unused "type: ignore" comment  [unused-ignore]

rclip (https://github.com/yurijmikhalevich/rclip)
- rclip/model.py:218: error: Argument "key" to "sorted" has incompatible type "Callable[[tuple[float64, int]], float64]"; expected "Callable[[tuple[float64, int]], SupportsDunderLT[Any] | SupportsDunderGT[Any]]"  [arg-type]
- rclip/model.py:218: error: Incompatible return value type (got "float64", expected "SupportsDunderLT[Any] | SupportsDunderGT[Any]")  [return-value]

hydpy (https://github.com/hydpy-dev/hydpy)
- hydpy/auxs/ppolytools.py:253: error: Value of type variable "_AnyShapeT" of "__call__" of "_ConstructorEmpty" cannot be "tuple[int, signedinteger[_64Bit]]"  [type-var]
- hydpy/auxs/ppolytools.py:253: error: Value of type variable "SupportsRichComparisonT" of "max" cannot be "signedinteger[_64Bit]"  [type-var]

ibis (https://github.com/ibis-project/ibis)
- ibis/expr/types/relations.py:2962: error: Value of type variable "SupportsRichComparisonT" of "sorted" cannot be "NumericValue | float"  [type-var]

scipy (https://github.com/scipy/scipy)
- scipy/spatial/tests/test_spherical_voronoi.py:194: error: Value of type variable "SupportsRichComparisonT" of "sorted" cannot be "floating[Any]"  [type-var]

arviz (https://github.com/arviz-devs/arviz)
- arviz/stats/ecdf_utils.py:72: error: Value of type variable "SupportsRichComparisonT" of "min" cannot be "Any | float64"  [type-var]

[srittau]

The primer diff will be interesting, once we merge #12939.

[srittau]

I don't think we'll get a consensus here, so I'm closing this in favor of the status quo.

[github-actions[bot]]

According to mypy_primer, this change has no effect on the checked open source code. 🤖🎉

[jorenham]

I don't think we'll get a consensus here

Why is that?

[randolf-scholz]

@srittau What happened with #12939?

[srittau]

I've reverted #12939, because as Jelle pointed out, SupportsBool is equivalent to object (as every object implements __bool__ via object).

[randolf-scholz]

@srittau It isn't. I even provided concrete examples in the OP of that PR.

[randolf-scholz]

I can add a test to the PR, ensuring that returning a plain object is not accepted.

[randolf-scholz]

@srittau I reopened as PR #14375, with an added test that checks that a plain object is not allowed, as well some test that check min against builtins. This also documents some known false negatives (which I do believe are not fixable atm).

cc @jorenham