ClusterODM icon indicating copy to clipboard operation
ClusterODM copied to clipboard

Downloading all.zip fails if larger than 2GB

Open rrowlands opened this issue 1 year ago • 3 comments

When using acls other than "public-read", and downloading the all zip from S3, as per task/uuid/download/all.zip, this S3.GetObject

https://github.com/OpenDroneMap/ClusterODM/blob/7c773493827aa8e7044878898cab1e1fa0ca8a0c/libs/proxy.js#L540

request fails if the size of the all.zip file is too large (larger than 2GB?), as per:

https://github.com/aws/aws-sdk-js/issues/2916

This error can be found in the cluster odm logs as:

error: Error encountered downloading object RangeError: The value "..." is invalid for option "size".

The AWS team recommends using multi-part downloads to avoid this error.

rrowlands avatar Jun 06 '23 23:06 rrowlands