cloudpathlib
cloudpathlib copied to clipboard
[WIP] Proof of concept fsspec integration
Implementation of FsspecClient and FsspecPath that work with an fsspec concrete filesystem implementation.
Follow up on #96
- New abstract
FsspecClient,FsspecPathclasses. Each registered implementation in fsspec.registry has subclasses autogenerated, e.g.,S3Path,GCSPath,HTTPPath. FsspecPathwill automatically dispatch to correct subclass, the same wayCloudPathdoes. This dispatching is isolated toFsspecPath;CloudPathstill only dispatches to original implementations.- Running environment variable
FSSPEC_MODE=1will replaceCloudPath's dispatching so that it will dispatch to fsspec subclasses instead of the normal ones.
- Running environment variable
- New extras.
[fsspec]will installfsspec.- Each extras from
fsspechas been copied to[fsspec-<name>], e.g.fsspec-s3fs. [fsspec-all]will install allfsspecextras.- original
[all]does not include any fsspec extras
🚀 Deployed on https://deploy-preview-109--gallant-agnesi-5f7bb3.netlify.app
Codecov Report
Merging #109 (ba3d9f3) into master (8b230c3) will decrease coverage by
12.3%. The diff coverage is15.7%.
@@ Coverage Diff @@
## master #109 +/- ##
=========================================
- Coverage 91.4% 79.1% -12.4%
=========================================
Files 8 12 +4
Lines 680 792 +112
=========================================
+ Hits 622 627 +5
- Misses 58 165 +107
| Impacted Files | Coverage Δ | |
|---|---|---|
| cloudpathlib/fsspec/__init__.py | 0.0% <0.0%> (ø) |
|
| cloudpathlib/fsspec/fsspecclient.py | 0.0% <0.0%> (ø) |
|
| cloudpathlib/fsspec/fsspecpath.py | 0.0% <0.0%> (ø) |
|
| cloudpathlib/fsspec/implementations.py | 0.0% <0.0%> (ø) |
|
| cloudpathlib/cloudpath.py | 89.8% <100.0%> (+0.1%) |
:arrow_up: |
See discussion in #96