umap icon indicating copy to clipboard operation
umap copied to clipboard

Fixing flaky failures in `test_umap_metrics.py`

Open crawlingcub opened this issue 3 years ago • 0 comments

Hi,

I tried running your tests by removing the seed setting lines (such as np.random_seed(SEED) in umap/tests/conftests.py. I observed that test test_grad_metrics_match_metrics in test_umap_metrics.py fails 25/500 times that I ran. Each time the assertion for the mahalanobis metric failed.

Changing the default decimal places from 6 to 5 makes sure that the test does not fail (without seeds). Please let me know if this change seems reasonable. If yes, then I can look into some other tests which also failed during this experiment.

If you have any other suggestions, I will be happy to look into it and/or include it in this PR. Please let me know if you need any other info.

Thanks!

crawlingcub avatar Sep 24 '21 06:09 crawlingcub