s5cmd
s5cmd copied to clipboard
s5cmd ls --region argument, and implied lists respect the cp argument
If you're doing a copy from one region to another with --no-clobber
there are a lot of failure modes with region resolution. The resolution order of ls is
Region detection While executing the commands, s5cmd detects the region according to the following order of priority: --source-region or --destination-region flags of cp command. AWS_REGION environment variable. Region section of AWS profile. Auto detection from bucket region (via HeadBucket API call). us-east-1 as default region.
So this is actually impossible if the region section of your aws profile is set (which you might want as a default). Without the config, it relied on HeadBucket calls, which can sometimes cause performance problems when making a lot of copies with s5cmd run
.
Ideally, s5cmd would take in a --region
argument, and s5cmd cp $SRC $DST --destination-region $REGION --no-clobber
would pass the --destination-region
flag to the internal list needed by --no-clobber
.