cloudpathlib

cloudpathlib copied to clipboard

Update _list_dir to have a "files only" filter

1

It came up in code review that doing _list_dir calls and then filtering files introduces extra network calls to check if each file is a file. We can likely do...

pjbull

Implement shutils equivalent (clutils) that has the same API but accepts AnyPath

4

Hi, thanks for your awesome library ! Have been searching for a way to use both local and cloud path seamlessy and here it is :D One particular problem I...

ClementPinard

Handle `mkdir` for cloud providers that support creating directories

9

S3 has an interesting situation with folders. Like other object stores like Azure, it has a flat structure, and when you upload a file to `a/b/c.txt` for example, it creates...

jayqi

Currently, our test suite installs all of the possible dependencies for every cloud provider. This means that errors in the code that occur when only one of the cloud provider...

pjbull

S3: Relying on load to check file existance if is file/dir causes high error rate on S3

5

Hello, Recently I started using cloudpath library to upload entire directories to S3, but suddenly I noticed a weird behavior that we got a high error rate at S3 (according...

fadikhou

RequestPayer option for S3

2

Hi, Thank you for the efforts on this package - I appreciate your work! I've got feature proposal because I stumbled about this today and dont think it is implemented....

gc-sommer

Investigate concurrency support

2

We probably want to be able to do things like downloads of many files in parallel. Async may help (#28) but some backends may be able to do things like...

pjbull

Document alternatives

2

We should document alternatives to cloudpathlib. The only one I've seen is https://github.com/liormizr/s3path One common way this is done is to have an `## Alternatives` section towards the bottom of...

jayqi

Speed up `is_dir` on entries returned by `iterdir` and `glob`

6

The `is_dir` check is fairly expensive, but at least for S3 and Azure when the entries were created as a result of the client's `_list_dir` method, you can tell for...

analog-cbarber

az:// URL scheme isn't really a URL because it is not unique

3

The existing 'az://' pseudo-URL scheme violates the principal of URLs in that it is not universal and unique. The 's3://' bucket syntax is valid because S3 buckets are universally unique....

analog-cbarber