binaryornot icon indicating copy to clipboard operation
binaryornot copied to clipboard

test_never_crashes require a too large example

Open vincentbernat opened this issue 7 years ago • 2 comments
trafficstars

Hey!

With hypothesis 3.44.1, I get this error:

======================================================================
ERROR: test_never_crashes (tests.test_check.TestDetectionProperties)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/build/binaryornot-0.4.4+dfsg/tests/test_check.py", line 214, in test_never_crashes
    def test_never_crashes(self, data):
  File "/usr/lib/python2.7/dist-packages/hypothesis/core.py", line 1002, in wrapped_test
    state.run()
  File "/usr/lib/python2.7/dist-packages/hypothesis/core.py", line 727, in run
    runner.run()
  File "/usr/lib/python2.7/dist-packages/hypothesis/internal/conjecture/engine.py", line 470, in run
    self._run()
  File "/usr/lib/python2.7/dist-packages/hypothesis/internal/conjecture/engine.py", line 868, in _run
    self.generate_new_examples()
  File "/usr/lib/python2.7/dist-packages/hypothesis/internal/conjecture/engine.py", line 760, in generate_new_examples
    HealthCheck.large_base_example
  File "/usr/lib/python2.7/dist-packages/hypothesis/internal/healthcheck.py", line 38, in fail_health_check
    raise FailedHealthCheck(message, label)
FailedHealthCheck: The smallest natural example for your test is extremely large. This makes it difficult for Hypothesis to generate good examples, especially when trying to reduce failing ones at the end. Consider reducing the size of your data if it is of a fixed size. You could also fix this by improving how your data shrinks (see https://hypothesis.readthedocs.io/en/latest/data.html#shrinking for details), or by introducing default values inside your strategy. e.g. could you replace some arguments with their defaults by using one_of(none(), some_complex_strategy)?
See https://hypothesis.readthedocs.io/en/latest/healthchecks.html for more information about this. If you want to disable just this health check, add HealthCheck.large_base_example to the suppress_health_check settings for this test.

Reducing the average size to 256 is not enough. 128 is OK. Dunno if there is a better way.

vincentbernat avatar Jan 18 '18 07:01 vincentbernat

We're getting the same error in builds for NixOS as well.

kragniz avatar Mar 04 '18 00:03 kragniz

@kragniz Please have a look at the staging branch. It was fixed there in NixOS/nixpkgs@4a9a7d30368c66af9bb5e2d95c21012badd9b699.

dotlambda avatar Mar 04 '18 09:03 dotlambda