lakeFS
lakeFS copied to clipboard
[Bug]: lakectl- Uploading large files (25GiB for instance) fails against AWS S3
What happened?
When using lakectl to upload a large file (25GiB was tested), to a lakeFS instance backed by an S3 bucket storage backend, the upload of a part of the multipart upload fails (against AWS S3) with the error:
part(400 Bad Request)
Steps to Reproduce:
lakectl fs upload lakefs://repo/branch/path -s <A large file>
Expected behavior
Upload should succeed regardless of object size.
lakeFS version
1.70
How lakeFS is installed
No response
Affected clients
No response
Relevant log output
part 997 upload https://<ACTUAL S3 BUCKET/PATH URL> part(400 Bad Request): request failed
Error executing command.
Command exited with non-zero status 1
Command being timed: "lakectl fs upload lakefs://repo/branch/path -s <A large file>"
Contact details
No response
@talSofer @Jonathan-Rosenberg is this issue still open i would like to contribute please assign this to me!