aws-sdk-js-v3
aws-sdk-js-v3 copied to clipboard
Add support for multipart download for S3 Transfer Manager
Describe the feature
Currently the JS SDK transfer manager package (@aws-sdk/lib-storage
) only supports an upload
function (based on PutObject). Ideally we should add support for an download
functionality that can take advantage of the range
option in getObject to achieve mutlipart functionality.
Use Case
This feature is crucial for applications that need to download large files or datasets from S3. Currently, developers must implement custom solutions to handle large file downloads, which can be error-prone and inefficient. Having a built-in download manager would simplify this process, making it more reliable and performant.
Proposed Solution
Use the existing design and functionality of the Go SDK v2 download manager as a reference. Implement a similar Downloader class in the JS SDK that supports multipart downloads, concurrency, error handling, and retry mechanisms. The downloader should allow configuring part size, concurrency level, and retry counts.
Other Information
No response
Acknowledgements
- [ ] I may be able to implement this feature request
- [ ] This feature might incur a breaking change
SDK version used
N/A
Environment details (OS name and version, etc.)
N/A