Jack Kelly

> To be sure, if you don't use gcs/fsspec in the main process, all is well? That's _almost_ the case in the minimal code snippet at the top of this...

> Right, but only with fork Yeah, exactly. I've had a quick look at the PyTorch library code and tried to create a simple minimal example. I've failed so far!...

Quick update: I've just tried opening my Zarr file on Google Cloud like this: ```python import xarray as xr dataset = xr.open_zarr('gs:///') ``` instead of like this: ```python gcs =...

Ah, sorry for not being explicit: I'm still using multiprocessing (via PyTorch), and the code still hangs if I use multiprocessing with gcsfs 2021.4.0 or 0.8, sorry. In terms of...

Hi @martindurant, Re-reading this issue, I'm starting to wonder if I'm doing something stupid in my code :) In my code, _every_ sub-process does this when the sub-process starts up:...

Oooh... some good news! I've got my multiprocessing code working with PyTorch and gcsfs 2021.4.0 :) I've made quite a few changes so I need to do some more tinkering...

I'm pretty sure it's the trick you suggested in [your comment above](#issuecomment-839929801that) which fixed the issue :) ```python fsspec.asyn.iothread[0] = None fsspec.asyn.loop[0] = None ``` I run that in every...

I've done a few more experiments (in the hopes that this might be of use to other people in a similar situation; or maybe useful to help understand what's going...

Yeah, I agree - I think it's sufficient to just document this somewhere. Happy to have a go at drafting a paragraph or two if you can recommend where best...

Awesome - thanks loads for running this experiment! Sounds like good news to me!