array-api-tests
array-api-tests copied to clipboard
Reset the max-examples default back to 100
The tests run fast enough now that there's no need to lower this to 20.
See the discussion at #246.
Could we just change our own workflow to have --hypothesis-max-examples=100
, instead of the default? As defo for something like JAX this makes it basically unusable on default runs :sweat_smile:
JAX is the exception, not us. If JAX is slower than most libraries, they should manually lower their max examples. Library defaults should match the common case, not the least common denominator.
I also changed the unvectorized max examples to scale linearly with max-examples instead of logarithmically. Otherwise, it's very difficult to actually run a lot of examples for unvectorized tests. We can adjust the scaling factor if 1/10 is not small enough by default.