desync icon indicating copy to clipboard operation
desync copied to clipboard

S3 transient tcp errors on large uploads

Open Joe-Improbable opened this issue 5 years ago • 0 comments

I've encountered a transient issue when performing large uploads to an S3 store.

I occasionally run into this error (perhaps once in 100 uploads):

Put "<s3 bucket name>/<chunk id>/<chunk id>.cacnk": write tcp <ip>-><ip>: use of closed network connection

A simple solution to this problem would just be to retry any failed PUT (or GET) requests, similar to how the RemoteHTTPStore does with error-retry option.

Minio-go implements retries but only for errors it deems retry-able (listed here), unfortunately any TCP errors won't be retried even though a number of them can be solved by retrying the request.

Joe-Improbable avatar Oct 28 '20 13:10 Joe-Improbable