s3sync icon indicating copy to clipboard operation
s3sync copied to clipboard

Unexpected EOF when synching bucket to local fs

Open temelt opened this issue 3 years ago • 4 comments

I am using the version 2.25, when i run the sync with these parameters

./s3sync -d --sync-log --sk **** --ss *** --se hostaddr -w 20 s3://test fs://./data/

it is working some time but after some downloads getting this error.

DEBU[0050] Recv pipeline err: object: )gFE5OB2/11.xo1dBK1OamWlMJsb.rnd sync error: unexpected EOF ERRO[0050] Sync error: pipeline step: 1 (LoadObjData) failed with error: object: )gFE5OB2/11.xo1dBK1OamWlMJsb.rnd sync error: unexpected EOF, terminating

Same for version 2.30 INFO[0901] Sync file Content-Type=application/octet-stream key=9zabR6AR/17.yXJzkzaeeS9gDKpn.rnd size=10485760 DEBU[0903] Recv pipeline err: object: 9Agq8lXt/82.Lit0v8qT2Ptof0H).rnd sync error: unexpected EOF ERRO[0903] Sync error: pipeline step: 1 (LoadObjData) failed with error: object: 9Agq8lXt/82.Lit0v8qT2Ptof0H).rnd sync error: unexpected EOF, terminating

and also that's not good to not have any details about the error, (debug log enabled, retry enabled, sync-log enabled )

Thanks in advance

temelt avatar Apr 29 '21 11:04 temelt

Hello. It's look like network/server issue. The network connection was dropped before read was completed. You can:

  1. Check S3 servers logs. Maybe server crashing and restarting or another server issue.
  2. Check network (packet loss or incorrect MTU).
  3. Try do download this files via curl or another http client.

cl-bvl avatar May 07 '21 16:05 cl-bvl

I've tried with RClone and all the files downloaded successfully (25GB/2500Files)

temelt avatar May 08 '21 12:05 temelt

Same issue here.

schickling avatar Oct 23 '21 09:10 schickling

Getting the same issue syncing out of backblaze.

andrewchambers avatar Jan 16 '23 10:01 andrewchambers