umap
umap copied to clipboard
Fixing flaky failures in `test_umap_metrics.py`
Hi,
I tried running your tests by removing the seed setting lines (such as np.random_seed(SEED)
in umap/tests/conftests.py
. I observed that test test_grad_metrics_match_metrics
in test_umap_metrics.py
fails 25/500 times that I ran. Each time the assertion for the mahalanobis
metric failed.
Changing the default decimal
places from 6
to 5
makes sure that the test does not fail (without seeds). Please let me know if this change seems reasonable. If yes, then I can look into some other tests which also failed during this experiment.
If you have any other suggestions, I will be happy to look into it and/or include it in this PR. Please let me know if you need any other info.
Thanks!