nolds
nolds copied to clipboard
Introduce regression tests with deterministic values
Since we already know that the output of some metrics changes slightly when we switch to numpy 2.0 (see #42), we should protect against this early on. The best way I see to catch this is to have regression tests and check for exact results on some sample data.
- [x] Create a regression test branching out from main. Just use random data for each of the main algorithms.
- [x] (Optional) check if there are any major branches in any of the algorithms that only trigger under specific conditions that are not met with the random data. (We can use the debugger for that.)
- [x] (Optional) also test with algorithms that include randomness by keeping seed fixed.
- [x] Store the exact values produced by the algorithms on this data.
- [ ] Document the tests.
- [ ] Merge tests into dev branch.
- [ ] Switch back to the poetry branch (using updated scikit-learn) and check if regression tests still run.