verified-sources
verified-sources copied to clipboard
support git filesystem in `filesystem` source
Which source is this for
adds git filesystem support to filesystem
source
Describe the data you'd like, be specific, link the docs
We want to support git in filesystem
source the same way we support buckets. This will involve using (and possibly reimplementing) the filesystem provided by fsspec by default.
-
- [ ] try if the default git filesystem implementation works, if not we'll need to implement our own. the problem I see is that last modification timestamp of the file in git repo is not returned by the default fsspec impl.
-
- [ ] allow both local and remote git repositories.
dlt
has a good support for cloning and checking out the branches: https://github.com/dlt-hub/dlt/blob/master/dlt/common/git.py we also have support forgiturlparse
for remot repos
- [ ] allow both local and remote git repositories.
-
- [ ] initially you can add all the code (including custom fsspec filesystem) to the
filesystem
verified source
- [ ] initially you can add all the code (including custom fsspec filesystem) to the
-
- [ ] if you need to modify library core, do it a fork from which we can install it for testing. after a review we'll merge it
-
- [ ] please reproduce all the test cases by creating a repo with the test files used by other filesystems in
filesystem
source
- [ ] please reproduce all the test cases by creating a repo with the test files used by other filesystems in