cloudpathlib

cloudpathlib copied to clipboard

Support Azure Data Lake Storage Gen2

12

Not sure if this problem also affects regular Blob storage accounts, but I have a Azure storage account using Data Lake Gen2 storage, and the I observe the following issues:...

analog-cbarber

Our CI tests currently only test the `[all]` dependency configuration. This fails to surface problems when missing optional dependencies cause runtime errors, e.g., the error fixed by #170. (Luckily, our...

jayqi

The current implementation does not intercept exceptions thrown by the various underlying cloud apis. As a result, it is up to the user to import those from the underlying libraries...

analog-cbarber

ENH: Parse cloud provider URLs into CloudPath objects

7

Hello, it's me again 😃 Could it be possible to manage cloud URLs the same way than CloudPaths (ie. `https://my-bucket.s3.us-west-2.amazonaws.com/puppy.png`) ? Or maybe just encapsulate them in a URLPath class...

remi-braun

Smarter default client lookup

1

Currently, the default client is constructed purely based on the implementation class and not on the path. It would be nice if there was a way to be able to...

analog-cbarber

Add ability to turn off local cache

3

Some use cases (especially if we enable streaming) may not want to keep much of a cache around at all. We should have an option to turn off the cache...

pjbull

Hello from fsspec!

18

I see that we have some goals in common. [fsspec](https://filesystem-spec.readthedocs.io/en/latest/) is a mature library in use by Dask, pandas, zarr and others. However, we *don't* have a good implementation of...

martindurant

If you're processing a lot of data in a loop and want to reclaim disk space after you're done with one path, then it could be helpful to have a...

jayqi

Add ability to detect if cache is dirty and should be uploaded even if file was not opened by cloudpathlib

4

We all want to live in a world where every Python library hands PathLike objects. This is not that world. Many libraries need a path to a local file—especially as...

pjbull

`shutil.copytree` has an `ignore` argument that lets one filter stuff out. Would be nice to support that too, but maybe it can be a future enhnacement. https://docs.python.org/3/library/shutil.html#shutil.copytree _Originally posted by...

pjbull