xarray icon indicating copy to clipboard operation
xarray copied to clipboard

test_weighted.test_weighted_operations_nonequal_coords should avoid depending on random number seed

Open shoyer opened this issue 3 years ago • 0 comments

What happened?

In testing an upgrade to the latest version of xarray in our systems, I noticed this test failing:

def test_weighted_operations_nonequal_coords():
        # There are no weights for a == 4, so that data point is ignored.
        weights = DataArray(np.random.randn(4), dims=("a",), coords=dict(a=[0, 1, 2, 3]))
        data = DataArray(np.random.randn(4), dims=("a",), coords=dict(a=[1, 2, 3, 4]))
        check_weighted_operations(data, weights, dim="a", skipna=None)
    
        q = 0.5
        result = data.weighted(weights).quantile(q, dim="a")
        # Expected value computed using code from [https://aakinshin.net/posts/weighted-quantiles/](https://www.google.com/url?q=https://aakinshin.net/posts/weighted-quantiles/&sa=D) with values at a=1,2,3
        expected = DataArray([0.9308707], coords={"quantile": [q]}).squeeze()
>       assert_allclose(result, expected)
E       AssertionError: Left and right DataArray objects are not close
E       
E       Differing values:
E       L
E           array(0.919569)
E       R
E           array(0.930871)

It appears that this test is hard-coded to match a particular random number seed, which in turn would fix the resutls of np.random.randn().

What did you expect to happen?

Whenever possible, Xarray's own tests should avoid relying on particular random number generators, e.g., in this case we could specify random numbers instead.

A back-up option would be to explicitly set random seed locally inside the tests, e.g., by creating a np.random.RandomState() with a fixed seed and using that. The global random state used by np.random.randn() is sensitive to implementation details like the order in which tests are run.

Minimal Complete Verifiable Example

No response

Relevant log output

No response

Anything else we need to know?

No response

Environment

...

shoyer avatar Apr 20 '22 19:04 shoyer