skyplane icon indicating copy to clipboard operation
skyplane copied to clipboard

[request] Enable multipart for on-prem transfers

Open ShishirPatil opened this issue 3 years ago • 0 comments

Please describe problem to be solved
The current on-prem implementation #735 does not have multipart transfers implemented. This is relatively straightforward for ext4fs/nfs, but will need some work for HDFS given our design choices. This is important to realize the potential benefits of parallel transfers on-prem.

ShishirPatil avatar Jan 30 '23 19:01 ShishirPatil