OceanParcels
`test_mitgcm` test failing with new dask release
docs/examples/example_dask_chunk_OCMs.py::test_mitgcm has been failing in CI this week with the release of dask v2024.11.0.
Downgrading to v2024.10.0 fixes the error, but it would be good to investigate and patch the test.
This has to do with the chunksize=auto-chunking. This is a quite tricky part of the code to comprehend; not sure it'll be easy to find why/what changed. Alternative is to relax the unit-test? The most important tests here are that the chunk_mode=specific* tests pass, which they still seem to do
https://github.com/OceanParcels/Parcels/blob/eb97b325bb782b3748f567e2ca45b32affb882e5/docs/examples/example_dask_chunk_OCMs.py#L564-L567
Yeah, I'm struggling to understand what the test (and underlying fieldfilebuffer.py) is doing. The fact that we have failure here:
if chunk_mode != "specific_different":
assert len(fieldset.U.grid._load_chunk) == len(fieldset.V.grid._load_chunk)
leads me to think that when using auto chunking that for some reason the U and V grids don't have the same number of chunks. Not sure what the implications of that are though.
@erikvansebille any immediate ideas how do you think we could relax the tests? I think these tests could do with a refactor down the line.
xref #853
@erikvansebille any immediate ideas how do you think we could relax the tests? I think these tests could do with a refactor down the line.
Yes, a refactor would be good down the line; also when we get uxarray-support in Parcels v4? But until then, perhaps we can just xfail these tests?