filesystem_spec
filesystem_spec copied to clipboard
fsspec
A specification that python filesystems should adhere to.
Here is the code to reproduce (fsspec version 2024.3.1): ```python import fsspec fs = fsspec.filesystem('local') with fs.transaction: with fs.open('/tmp/a.txt.gz', 'wt', compression='infer') as f: f.write('Hello, world!\n') ``` Here is the error:...
I am attempting to use the `DirFileSystem` to wrap `SMBFileSystem` so I can operate in my SMB folder as though it's my base directory. I can use most methods I...
Path expansion is inconsistent when sequences are passed in to helper utilities such as `fsspec.mv`, `fsspec.copy` etc... `fsspec.open_files()` will call expand_if_needed depending on the args, but `fsspec.mv` and `fsspec.copy` will...
Hi, I'd like to use FTP with SSL/TLS in fsspec, to be specific: [this implementation](https://docs.python.org/3/library/ftplib.html#ftp-tls-objects). I've made a PR for it, will post the link soon. Regards, Bart
Hi all. Love fsspec. I'm trying to use GenericFileSystem like this: ```python import fsspec import fsspec.generic fs = fsspec.url_to_fs("sftp://username@host")[0] fsspec.generic.rsync( "sftp:///stuff", # only the path necessary here. the username and...
In our project, we use the GenericFileSystem's [`_cp_file`](https://github.com/fsspec/filesystem_spec/blob/012816bb142de1e507423b81dc3b2c925d4424c3/fsspec/generic.py#L254) function to (asynchronously) copy files from a remote SSH filesystem to our local filesystem (as suggested in [this comment](https://github.com/fsspec/filesystem_spec/issues/909#issuecomment-1204212507)). Before the release...
Dear all, I'm struggling finding out why this is not working When I use caching on a MemoryFileSystem, fsspec raises a FileNotFoundError with the ``open`` method. Here is a reproducible...
Hello, I noticed that the `LocalFileSystem._strip_protocol` signature changed in #1477, when running the universal_pathlib test-suite against the current fsspec version. To me it seems that the intention of #1477 was...
The system tries to connect 5 times in a row, ignoring all errors. Since the for loop has no `else` branch, the function does not raise any error when all...
Hello, ## Summary I initially created an issue on the xarray repository: https://github.com/pydata/xarray/issues/8944 ; I place it here for reference as it provides more context. The part concerning `fsspec` is...
