giotto-tda icon indicating copy to clipboard operation
giotto-tda copied to clipboard

test_projection_values_equal_slice failed due to unreliable test timings

Open rth opened this issue 5 years ago • 1 comments

test_projection_values_equal_slice failed on master in fcffaf8 due to unreliable test timings in hypothesis,

=================================== FAILURES ===================================
______________________ test_projection_values_equal_slice ______________________

>   ???

self = <hypothesis.core.StateForActualGivenExecution object at 0x7fd0e5a29430>
message = 'Hypothesis test_projection_values_equal_slice(X=array([[0.]])) produces unreliable results: Falsified on the first call but did not on a subsequent one'

    def __flaky(self, message):
        if len(self.falsifying_examples) <= 1:
>           raise Flaky(message)
E           hypothesis.errors.Flaky: Hypothesis test_projection_values_equal_slice(X=array([[0.]])) produces unreliable results: Falsified on the first call but did not on a subsequent one

/opt/hostedtoolcache/Python/3.8.1/x64/lib/python3.8/site-packages/hypothesis/core.py:835: Flaky
---------------------------------- Hypothesis ----------------------------------
Falsifying example: test_projection_values_equal_slice(
    X=array([[0.]]),
)
Unreliable test timings! On an initial run, this test took 632.78ms, which exceeded the deadline of 200.00ms, but on a subsequent run it took 0.34 ms, which did not. If you expect this sort of variability in your test timings, consider turning deadlines off for this test by setting deadline=None.
=============================== warnings summary ===============================

likely not critical, still the different in run time is surprising.

rth avatar Feb 20 '20 10:02 rth

Thanks! Yes, we noticed that before but it slipped due to other priorities, good it's now here so we don't forget!

ulupo avatar Feb 20 '20 10:02 ulupo