filesystem_spec

filesystem_spec copied to clipboard

Expected behavior for "directory to new directory" depends on trailing slash

5

In [Copying files and directories](https://filesystem-spec.readthedocs.io/en/latest/copying.html#single-source-to-single-target), 1f case "directory to new directory", it is stated that the trailing slashes in both `source` and `target` are optional. However, if `source/subdir` and `target/newdir/`...

Github-dm-CDE

Continuation of https://github.com/fsspec/filesystem_spec/pull/1284, but add type hints to `fsspec.mapping` and `fsspec.dircache`.

alanhdu

http get() call creates malformed dest path

1

See original discussion [here](https://github.com/fsspec/filesystem_spec/discussions/1490) and a linked bug [here](https://github.com/fsspec/filesystem_spec/issues/1504) ### Issue Creating an http file system and using an `rpath` with query params results in a malformed destination path. ###...

cgrass

order dependent file listing

5

Have you guys seen something like this before? ``` @pytest.fixture def source_folder(): # Set up source directory for testing parent_temp = tempfile.mkdtemp() src_dir = os.path.join(parent_temp, "source", "") nested_dir = os.path.join(src_dir,...

wild-endeavor

ArrowFSWrapper should not use "/" root_marker for all filesystems

3

Perhaps a bit too hacky to be merged, but Fixes https://github.com/fsspec/filesystem_spec/issues/1464 , in theory. Basic idea is to check if the provided arrow filesystem supports a `/` path, then create...

b-phi

429 Client Error: Too Many Requests for url azuredatabricks.net/api/2.0/dbfs/mkdirs

9

I'm using the "fsspec.implementations.dbfs import DatabricksFileSystem" with pyarrow to write a parquet dataset on the DatabricksFilesystem DBFS, but when using the DatabricksFilesystem implementation with the write_to_dataset method from pyarrow I'm...

a24lorie

Reading data from `raw.githubusercontent.com` hangs

2

## What When using the GitHub `fsspec` implementation, an attempt to `open(...)` data at `raw.githubusercontent.com` hangs indefinitely on my local machine. ## Why My DNS server returns both IPv4 and...

nenb

2023.12.0: pytest fails in units which are using `libarchive` module

1

I'm packaging your module as an rpm package so I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account. - `python3 -sBm...

kloczek

ArrowFSWrapper should not use "/" root_marker for all filesystems

1

For the arrow S3FileSystem for example, using `root_market = "/"` throws an error when trying to upload a local file. Love the arrow integration, please let me know if I'm...

b-phi

Poor performance of async IO in a multithreaded env

10

Thanks for the great work! I experience poor performance of IO bound tasks in a multithreaded environment. Dask based example: ```python import fsspec import dask.array as da import xarray as...

ravwojdyla