filesystem_spec icon indicating copy to clipboard operation
filesystem_spec copied to clipboard

API for conditional / exclusive write

Open TomAugspurger opened this issue 1 year ago • 3 comments

Over in https://github.com/zarr-developers/zarr-python/pull/2262, we'd like to write a file but only if it doesn't already exist. On a local file system, this would be open(path, mode="xb"), which will fail with a FileExistsError if the file already exists.

Now that S3 supports conditional writes, it should be possible to implement this for s3fs, gcsfs (if_generation_match=0), and adlfs (overwrite=False).

Would there be any appetite for standardizing this behavior? I'm not sure what API is best, but I lean towards something like an overwrite: bool parameter to pipe and similar methods. We could also try to support mode=xb in some open-like methods, but I'm less sure about that.

TomAugspurger avatar Sep 27 '24 01:09 TomAugspurger