filesystem_spec
filesystem_spec copied to clipboard
fsspec
A specification that python filesystems should adhere to.
I'm using fsspec to load a zip file which store in HDFS, but encounter a problem ``` python3.9/fsspec/implementations/zip.py", line 54, in __init__ self.zip = zipfile.ZipFile(self.fo, mode=mode) File "python3.9/zipfile.py", line 1257,...
Preserves permissions when copying local file. Fixes #1524
Hi, I am trying to copy a local file with the following permissions: ``` -rwxrwxr-x 1 rhorvath rhorvath 7994 ``` after copying the file it becomes: ``` -rw-rw-r-- 1 rhorvath...
A file system's corresponding transaction class has been a class attribute since https://github.com/fsspec/filesystem_spec/pull/1424. However, the `fsspec.transaction.Transaction` interface is currently stateless. For someone trying to roll a transaction class that takes...
Draft based on what we have in dvc, but I'm unhappy with how we try to work with both posixpath and ntpath in the base filesystem, so this is due...
These are some early thoughts, but I just wanted to share in case anyone else will find this useful or will have any thoughts/comments. Some fsspec filesystems might raise `EROFS`...
### Discussed in https://github.com/fsspec/filesystem_spec/discussions/1490 Originally posted by **cgrass** January 4, 2024 Hello, I'm new to python and fsspec, so hopefully there is an obvious answer to my question. Thanks for...
I have been debugging an issue for a while and finally have a small reproducible snip. Using `fsspec==2023.12.2` ```python import fsspec some_public_url = "http://replace.me.with.a.public.url" def read_my_data(url: str) -> None: for...
In https://github.com/fsspec/filesystem_spec/blob/7b774c26e30e5ae60b5b63bf4e345a1b32028a4e/fsspec/core.py#L337C9-L337C41 the dictionaries `extra_kwargs` and `kws` are merged with `kw = dict(**extra_kwargs, **kws)`. This may be an issue when there is redundant information in the two dictionaries, resulting in...
The unit tests for get()/put()/copy() did not cover all combinations of input parameters.
