filesystem_spec

filesystem_spec copied to clipboard

AbstractFileSystem.exists() catches every exception

1

Hello, I would like to kindly request a change in the implementation of `AbstractFileSystem.exists()`, as it currently catches every exception that might get thrown without any notice. An in my...

tfelbr

Using callbacks to filter files?

1

Just a question - I would like to use the recursive mechanisms in fsspec to copy directories, but I also want to exclude some paths (for example `.git` directories). It...

hhuuggoo

Proposal to change implementation of globbing for possible performance increase

9

Hello, I am using the build-in SFTP implementation to access an SFTP server. Unfortunately, this server is pretty slow, so I used globbing with the hope it will speed up...

tfelbr

make `fsspec.asyn._get_batch_size()` public

5

Being able to get the configured or system default batch size from fsspec is useful when using filesystems that support an additional level of concurrency (beyond the existing fs methods...

pmrowla

Open the http file system in the wb mode

1

We can only use rb mode in the HTTP file system for now. It will be great if we can support wb mode. https://github.com/fsspec/filesystem_spec/blob/61cdf52cd0aaf524945ffb83502be39fccaee363/fsspec/implementations/http.py#L350-L351

pingsutw

According to the [documentation of AbstractFileSystem](https://github.com/fsspec/filesystem_spec/blob/master/fsspec/spec.py#L541:L542), `glob` should act as `ls` when the path ends with `/`. But when using the `LocalFileSystem` method, [the trailing slash is removed](https://github.com/fsspec/filesystem_spec/blob/master/fsspec/implementations/local.py#L69) when `_strip_protocol`...

john-jam

feature: size limit on caching filesystem

4

It would be nice to set a limit on the size of the caching filesystems. This would truly enable 'set and forget' kind of work instead of having to manually...

majidaldo

LocalFileSystem.rm(...) raises an exception when deleting a symlink of a directory

2

## Versions `fsspec==2023.6.0` ## Summary The check of `recursive` flag introduced by #1276 causes errors when `LocalFileSystem.rm(...)` tries to delete a symlink of a directory. ## POC Create a sample...

sugibuchi

FileNotFound for s3 file with / in the key

11

Hello! I am trying to read a csv stored in a remote s3 bucket with a / in the name, like this: `'s3://mybucket/path/7/update//nicktorba.part_00000'` When i run this code, I get...

nick-amplify

Feature request: classmethod returning possible keys of the details=True dictionary for each filesystem.

2

I have written a Qt ItemModel for fsspec, what is tricky though is that there is no way to know which keys the "details" dictionary can contain for each filesystem...

phil65