FGEM icon indicating copy to clipboard operation
FGEM copied to clipboard

Example K- Reference-FS's target filesystem must have same value of asynchronous

Open amsler-M opened this issue 4 months ago • 0 comments

Summary of the Issue:

You are running an FGEM example that loads weather data using Sup3rCC and zarr/fsspec. During the creation of the World object, a ValueError is raised. The error occurs when zarr.open(mapper) is called in your weather data loading function. This is triggered by a mismatch in the asynchronous property between two fsspec filesystem objects (the reference filesystem and the target S3 filesystem). This is a known compatibility issue between recent versions of fsspec, kerchunk, and zarr.

The code is trying to open a reference file system (likely S3 or cloud storage) using fsspec and zarr. The reference filesystem and the target filesystem must both be either asynchronous or synchronous, but in your case, one is async and the other is not. This is often caused by incompatible or mismatched versions of zarr, kerchunk, and fsspec, or by not explicitly setting the asynchronous parameter when creating the filesystem. How to resolve:

Ensure all relevant packages are up to date and compatible: zarr>=3.0.1 kerchunk>=0.2.4 fsspec>=2024.3.1 s3fs>=2024.3.1 If you must use zarr==2.18, you cannot use the latest kerchunk or fgem (see previous summary). Explicitly set asynchronous=False in your code when creating the S3 filesystem in fsspec.get_mapper

amsler-M avatar Aug 05 '25 06:08 amsler-M