s5cmd icon indicating copy to clipboard operation
s5cmd copied to clipboard

Frequent signature expiry during sync of large files w/ v2.0.0 (but not v2.0.0-beta.2)

Open andreas-venturini opened this issue 2 years ago • 1 comments

We run s5cmd sync on a folder w/ ~20 TByte of video files (around ~250k files in total). With the latest v2.0.0 release the process starts returning expired signature errors sooner or later (sometimes only a couple new files are uploaded before all requests return the expiry warning):

request expired, resigning

This never happens with v2.0.0-beta2.

We run the following command against a target bucket in AWS eu-west-1 region:

s5cmd sync \
  --no-follow-symlinks \
  --exclude "*/temp/*" \
  --acl "private" \
  --sse "AES256" \
  --storage-class "GLACIER_IR" \
  $LOCAL_SOURCE $S3_TARGET

andreas-venturini avatar Oct 20 '22 16:10 andreas-venturini

Very strange. When I check the diff between v2.0.0-beta against v2.0.0, there's no logical change that could result in an error.

igungor avatar Jul 25 '23 13:07 igungor