url_to_fs for a zip file system on a s3 bucket stopped working after upgrade.

[louis-van-der-stam]

fsspec.mapping.url_to_fs("zip::s://some-bucket/some-file.zip", mode="w") stopped working somewhere between 2023.5.0 and 2023.9.2.

It seems to have something to do with removing the "mode" argument as one of the first things in the function.

Is this intended or an unexpected side effect?

[martindurant]

Ah, "mode" was one of the parameters which were assumed to be used in open() but not for configuring a filesystem, which is clearly wrong. Does using

url_to_fs("zip::s://some-bucket/some-file.zip", zip=dict(mode="w"))

work?

[louis-van-der-stam]

Yes, that works. But how does that work? Is it because the name of the kwarg matches the name of the protocol? And could we also add additional parameters for s3 that way so add s3={k : v} or should those be k=v kwargs because s3 is the "inner" protocol?

We actually fixed this by doing: ZipFileSystem("s://some-bucket/some-file.zip", mode="w"). Is that equally good or?

By the way, thanks for creating and maintaining the library, never thought changing from writing several files to an s3 bucket to writing a single zip to an s3 bucket would be this simple!

[martindurant]

Sorry, forgot to answer here.

Is it because the name of the kwarg matches the name of the protocol

exactly

could we also add additional parameters for s3 that way so add s3={k : v}

yes

should those be k=v kwargs because s3 is the "inner" protocol

no, they would go to zip as the outer protocol; but mode had a special meaning already from the more general open().