s5cmd icon indicating copy to clipboard operation
s5cmd copied to clipboard

s5cmd ls --region argument, and implied lists respect the cp argument

Open jordan-jack-schneider opened this issue 1 year ago • 0 comments

If you're doing a copy from one region to another with --no-clobber there are a lot of failure modes with region resolution. The resolution order of ls is

Region detection While executing the commands, s5cmd detects the region according to the following order of priority: --source-region or --destination-region flags of cp command. AWS_REGION environment variable. Region section of AWS profile. Auto detection from bucket region (via HeadBucket API call). us-east-1 as default region.

So this is actually impossible if the region section of your aws profile is set (which you might want as a default). Without the config, it relied on HeadBucket calls, which can sometimes cause performance problems when making a lot of copies with s5cmd run.

Ideally, s5cmd would take in a --region argument, and s5cmd cp $SRC $DST --destination-region $REGION --no-clobber would pass the --destination-region flag to the internal list needed by --no-clobber.

jordan-jack-schneider avatar Feb 14 '24 20:02 jordan-jack-schneider