nowcasting_dataset
nowcasting_dataset copied to clipboard
openclimatefix
Discussion: Should we be using random numbers in our tests?
(not urgent.. this can wait until 2022 :slightly_smiling_face: )
I fear that using random numbers in our tests could lead to intermittent (and hence hard-to-debug) unittest failures.
Let's not worry about this now, though! We can discuss later...
Yea agree with can deal with it later.
Pro: Part of me likes having random numbers as it is more likely to cover cases that we havent thought of
Con: as you say above - can be a problem with hard-to-debug failures
I think this unit test failure just now in the github CI (that I haven't seen before, and has nothing to do with the PR) might be to do with random numbers!
interesting discuss on the pros and cons of random numbers in tests: https://stackoverflow.com/questions/32458/random-data-in-unit-tests
Perhaps 'seeding' them is a good comprise?
yeah, sounds good to me!
I'd guess we'd need to use the new(ish) np.Generator, one Generator per test (maybe?) to guarantee that the test data remains the same, no matter what order the tests are run in