zarr-python icon indicating copy to clipboard operation
zarr-python copied to clipboard

use our fork of s3fs for upstream testing

Open d-v-b opened this issue 7 months ago • 4 comments

s3fs depends on an exact version of fsspec (2025.5.1). The unreleased version of fsspec, i.e. what you get when you install fsspec from a git dependency declaration like fsspec @ git+https://github.com/fsspec/filesystem_spec, is generally ahead of that version, and thus the two are, by declaration, incompatible. This apparent incompatibility has prevented recent upstream tests from succeeding.

As a short-term fix, I forked s3fs and relaxed the upper bound of its fsspec dependency. This PR changes the upstream tests s3fs dependency to use our fork.

this was discussed over at s3fs here

d-v-b avatar May 24 '25 12:05 d-v-b

because the impact of this PR is narrowly scoped to a subsection of our CI, and it works at its intended purpose, I'd like to merge this soon and without release notes.

d-v-b avatar May 24 '25 12:05 d-v-b

This is working again now: https://github.com/zarr-developers/zarr-python/actions/runs/15230896040/job/42838622250?pr=2874, so no need for this.

I think the failure was hitting an edge case where one of the git commits was tagged with a specific tag, so the version of the package wasn't a dev version. I haven't investigated thoroughly though to confirm this.

dstansby avatar May 25 '25 09:05 dstansby

This is working again now: https://github.com/zarr-developers/zarr-python/actions/runs/15230896040/job/42838622250?pr=2874, so no need for this.

I think the failure was hitting an edge case where one of the git commits was tagged with a specific tag, so the version of the package wasn't a dev version. I haven't investigated thoroughly though to confirm this.

Nice, I will close this.

d-v-b avatar May 25 '25 09:05 d-v-b

I'm going to re-open this, because we are seeing problems with our upstream tests again in #2774

d-v-b avatar Jun 04 '25 12:06 d-v-b

closing this as unnecessary, we can open it again (again) if we run into these issues again

d-v-b avatar Sep 18 '25 13:09 d-v-b