filesystem_spec
filesystem_spec copied to clipboard
fsspec
A specification that python filesystems should adhere to.
In the `memory` filesystem there is an error if you try to open it with `r+b` mode saying this mode is not supported. However this mode is apparently supported (https://github.com/fsspec/filesystem_spec/blob/master/fsspec/implementations/memory.py#L178)....
The CachingFileSystem can benefit from calling `cat_ranges` for asynchronous filesystems to concurrently download blocks. This speeds up filling the cache significantly for filesystems like HTTP and S3FS.
fsspec.mapping.url_to_fs("zip::s://some-bucket/some-file.zip", mode="w") stopped working somewhere between 2023.5.0 and 2023.9.2. It seems to have something to do with removing the "mode" argument as one of the first things in the function....
Hi @martindurant, I'm currently fixing `memory` URI handling in universal_pathlib for python versions up to 3.11 and was wondering which URI would be considered the canonical form of a memory...
Apologies if this is currently possible, I couldn't find a clean way to achieve this. I think it would be useful to have a way to check if a given...
Addressing #1398 Pretty much identical to the generic one but doesn't create the directories async. The generic one defines a make_many_dirs which tries to make them async if it can...
The rsync method looks like exactly what I have been looking for, but I am not sure how one would use it to say sync data from two different s3...
What I wanted to do: Read and write to an Azure BlobStorage via pre-signed URLs using the fsspec http implementation. How I tried to do it: As fsspec does not...
A common access pattern is ``` with fsspec.open(...) as f: process(f.read()) ``` For AbstractBufferedFiles, this causes an initial info() call to figure out the length, but then we do the...