lakeFS
lakeFS copied to clipboard
treeverse
lakeFS - Data version control for your data lake | Git for data
Add an example of using fsspec with lakefs-spec in the DuckDB integration page
As part of https://github.com/treeverse/lakeFS/pull/7842, we noticed that we currently do not address directory markers in lakectl local. Need to understand if and how we should handle it.
Currently when users in S3 are uploading objects > 5GB from the HL Python SDK, they will receive an error due to S3 [limitations](https://aws.amazon.com/s3/faqs/#:~:text=The%20total%20volume%20of%20data,single%20PUT%20is%205%20GB.) on single object upload size. Since...
## What? Let's add an API to return the last object with a given prefix! For instance, if I have objects with these keys: ``` /aa/000/x /ab/000/y /ab/001/abc /ab/001/def /ab/001/xyz...
### What happened? Related to #7451 Steps to Reproduce: Running this code: ```python import lakefs lakefs.Repository("").branch("branch-name").object("something").exists() ``` Throws: ``` ... lakefs.exceptions.BadRequestException: code: 400, reason: Bad Request, body: {} ``` While...
We used to specify `latest` as the goreleaser's version, which by itself is a bad practice, but now it has introduced breaking changes (version 2). We should: 1. Upgrade goreleaser's...
Bumps [ws](https://github.com/websockets/ws) from 8.16.0 to 8.17.1. Release notes Sourced from ws's releases. 8.17.1 Bug fixes Fixed a DoS vulnerability (#2231). A request with a number of headers exceeding the[server.maxHeadersCount][] threshold...
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot] avatar"){kind=avatar}
Bumps [rexml](https://github.com/ruby/rexml) from 3.2.6 to 3.3.0. Release notes Sourced from rexml's releases. REXML 3.3.0 - 2024-06-11 Improvements Added support for strscan 0.7.0 installed with Ruby 2.6. GH-142 Reported by Fernando...
Bumps [braces](https://github.com/micromatch/braces) from 3.0.2 to 3.0.3. Commits 74b2db2 3.0.3 88f1429 update eslint. lint, fix unit tests. 415d660 Snyk js braces 6838727 (#40) 190510f fix tests, skip 1 test in test/braces.expand...
DBIO (DataBricks) performs many getFileStatus calls over lakeFSFS. Each of these calls looks for an object _or directory marker_ named `_started_*` or `_committed_*`. Looking for a marker involves a getObject,...