Missing use cases for binops on array vs. scalar

[crusaderky]

After reading hh.scalars, I understand that array-api-tests currently never generates the use cases of a binop where one argument is an Array with float or complex dtype and the other argument is

• a pure-python integer;
• a scalar inf, -inf, or nan;
• a scalar outside of the range [1/64, 64]

Unless I missed something and the tests for these cases are somewhere else?

https://github.com/data-apis/array-api-tests/blob/c1e5cda7836963e58e40bcdf68881a569b8c64d5/array_api_tests/hypothesis_helpers.py#L451-L481

https://github.com/data-apis/array-api-tests/blob/c1e5cda7836963e58e40bcdf68881a569b8c64d5/array_api_tests/hypothesis_helpers.py#L590-L597

Note that the [1/64, 64] range exclusively applies to float/complex. The code could use a clarification there as it is not obvious upon first read that integers ignore it.

e.g. pytest -k minimum returns this:

array_api_tests/test_has_names.py::test_has_names[elementwise-minimum] PASSED                                                                                    [ 20%]
array_api_tests/test_operators_and_elementwise_functions.py::test_minimum PASSED                                                                                 [ 40%]
array_api_tests/test_operators_and_elementwise_functions.py::test_binary_with_scalars_real[minimum] SKIPPED (requires ARRAY_API_TESTS_VERSION >= 2024.12)        [ 60%]
array_api_tests/test_signatures.py::test_func_signature[minimum] PASSED                                                                                          [ 80%]
array_api_tests/test_special_cases.py::test_binary[minimum(x1_i is NaN or x2_i is NaN) -> NaN] PASSED   

To my understanding

• in test_binary_with_scalars_real, when one arg is Array[float64] or Array[float32], the other arg is always in the [1/64, 64] range;
• test_special_cases::test_binary only tests arrays containing NaN, not scalar NaN;
• none tests inf or arrays containing inf

related:

• https://github.com/data-apis/array-api-tests/pull/360
• https://github.com/data-apis/array-api/issues/925

[ev-br]

In test_binary_with_scalars_real, when one arg is Array[float64] or Array[float32], the other arg is always in the [1/64, 64] range;

Yes. This is a shortcut of course, made to get some testing going.

There are several irksome bits to take into account when pondering about undoing it:

• Hypothesis likes to generate edge cases. If you don't limit the range, you are bound to get denormal numbers, values close to the min/max values etc.
• There is no way to xfail a part of a test. If a test fails on backend X because of an edge case around FLOAT_MAX, we have to xfail the whole test for this backend.

In general, value testing is very fiddly.

• test_special_cases needs separate attention, https://github.com/data-apis/array-api-tests/issues/284

---

My strategy so far was to first get test some coverage for 2024 additions, however rough; and use https://github.com/data-apis/array-api-tests/issues/299 and https://github.com/data-apis/array-api-tests/issues/301 to track the holes.