choicemodels icon indicating copy to clipboard operation
choicemodels copied to clipboard

Improving probabilistic tests

Open smmaurer opened this issue 6 years ago • 1 comments

Some of our unit tests are probabilistic, so they sometimes fail.

The tests are intended to verify that things like sampling weights are being applied correctly. But since the sampling is random, occasionally the tests don't pass. We should make this more sophisticated :slightly_smiling_face:

Here's an example of a probabilistic test: https://github.com/UDST/choicemodels/blob/master/tests/test_mct.py#L46

And a failure that was resolved simply by re-running the test: https://travis-ci.org/UDST/choicemodels/jobs/427893157

smmaurer avatar Sep 12 '18 22:09 smmaurer

I wrote a simple context manager that manages the numpy and python random states. Might be applicable here:

https://github.com/AZMAG/smartpy_core/blob/master/smartpy_core/wrangling.py#L12 https://github.com/AZMAG/smartpy_core/blob/master/smartpy_core/tests/test_wrangling.py#L163

bridwell avatar Sep 13 '18 23:09 bridwell