Jack Kelly

No worries, sounds good. When you do start the process of stabilising the `CodecPipeline` API, please could you tag me in the discussion? I'll probably just passively listen to the...

This sounds really interesting! We have a similar use-case as the use-case outlined in the original post... This might be a bit off-topic but perhaps it's useful to give some...

Just to connect up two related conversations... Over in [the "Zarr Benchmarking & Performance" group](https://github.com/zarr-developers/zarr-python/discussions/1479), we've briefly discussed (amongst other things) batched async IO and decompression (possibly implemented in Rust)....

I'll try to do this later this afternoon using `means = ds_from_zarr['stacked_eumetsat_data'].mean(dim=['time', 'x', 'y']).compute()`

OK, I've written a little script to get the means and mins... Just ignoring the `eumetsat_zarr_2020_02.zarr` (and not ignoring the "rectangle of zeros") the new stats are: ``` Coordinates: *...

The "rectangle of zeros" starts at 1,037,047 meters (OSGB "northing") and extends north. So need to ignore everything north of this

Wow - thank you for the very quick reply! You're right, adding the line `multiprocessing.set_start_method('forkserver')` or `multiprocessing.set_start_method('spawn')` fixes the issue in my minimal example... ...Let me check if this also...

Unfortunately setting the start method to either `spawn` or `forkserver` doesn't work for [my PyTorch code](https://github.com/openclimatefix/predict_pv_yield_2/blob/master/notebooks/21.0_include_PV_data.ipynb). Here's the error when I add `multiprocessing.set_start_method('spawn')` at the start of the code: ```python...

Thanks loads for the additional help! Unfortunately, PyTorch doesn't log error messages from the child processes. I've tried (briefly) and I can't seem to peer into the child processes (at...

Unfortunately this is broken for me in the two most recent versions of gcsfs (i.e. it's broken in both 0.8.0 and 2021.04.0).