Stateful tests with Dataset

[dcherian]

I was curious to see if the hypothesis stateful testing would catch an inconsistent sequence of index manipulation operations like #8646. Turns out rename_vars is basically broken? (filed #8659) :P

PS: this blog post is amazing.

E           state = DatasetStateMachine()
E           state.assert_invariants()
E           > ===
E
E            <xarray.Dataset>
E           Dimensions:  ()
E           Data variables:
E               *empty*
E           ===
E
E
E           > vars: ('1', '1_')
E           state.add_dim_coord(var=<xarray.Variable (1: 1)>
E           array([0], dtype=uint32))
E           state.assert_invariants()
E           > ===
E
E            <xarray.Dataset>
E           Dimensions:  (1: 1)
E           Coordinates:
E             * 1        (1) uint32 0
E           Data variables:
E               1_       (1) uint32 0
E           ===
E
E
E           > renaming 1 to 0
E           state.rename_vars(newname='0')
E           state.assert_invariants()
E           > ===
E
E            <xarray.Dataset>
E           Dimensions:  (1: 1)
E           Coordinates:
E             * 0        (1) uint32 0
E           Dimensions without coordinates: 1
E           Data variables:
E               1_       (1) uint32 0
E           ===
E
E
E           state.teardown()

[TomNicholas]

Oh this is a really cool idea!

Also yeah I've read that blog post before and my mind was also blown haha

[max-sixty]

Great idea! Really cool.

I even wonder to what extent this could replace vast swathes of our tests — anything where the output is calculated from the input.

We'll always want some unit tests to code to when developing, but plausibly the unit tests are either a) examples or b) some saved cases that have failed these model-based tests before.

[benbovy]

That looks very cool indeed! (although I don't think that rename_vars is broken :) but instead some valid cases require skipping the default indexes invariant check).

[dcherian]

instead some valid cases require skipping the default indexes invariant check.

Can we write down rules for when these checks are needed?

[benbovy]

Yes although I think it is easier to write rules for when these checks aren't needed.

One example: skip the default indexes invariant test when the name of an existing dimension coordinate is passed as input kwarg or dict key to .rename_vars().

[dcherian]

This seems to have found at least 4 sort-of-edge-case failures for swap_dims:

1. swapping a dimension to itself
2. swapping from Index to a MultiIndex level
3. swapping from MultiIndex to a level of the same MultiIndex
4. swapping from Index to MultiIndex variable.

[dcherian]

Hah now our strategy tests are failing

[dcherian]

Thanks for the review @Zac-HD