soto-s3-file-transfer icon indicating copy to clipboard operation
soto-s3-file-transfer copied to clipboard

slow upload speed

Open masaldana2 opened this issue 3 years ago • 5 comments

Hello This used to upload fast, now my users and I get 50kb max did something changed? I was having timeout issues which I solved using ,timeout: .minutes(5)

masaldana2 avatar Nov 10 '21 12:11 masaldana2

Last release was 2 months back. Over what time period have you seen the change?

adam-fowler avatar Nov 10 '21 14:11 adam-fowler

I just ran some of the tests and am not seeing any slowdown. Try running testMultipartUploadDownload to see what you get.

Also what platform are you running this on? I guess also are you uploading loads of small files, or a smaller number of large files

adam-fowler avatar Nov 10 '21 15:11 adam-fowler

I am also exploring slow uploads. How do I set timeout?

geoffhufford avatar Dec 02 '21 16:12 geoffhufford

Hi @geoffhufford ,

I have a few questions that might help me diagnose your issue

  • what platform are you running on
  • what version of Swift are you running?
  • Are you seeing slowdown when uploading lots of small files, or fewer large files. Are many of your files larger than 5MB?

Can you run the test I mention above to see if you can replicate your issue within that test?

S3.init has a parameter timeout. You can also use the with function which will take an existing S3 object and set the timeout for it

let newS3 = s3.with(timeout: .seconds(120))

adam-fowler avatar Dec 02 '21 17:12 adam-fowler

@adam-fowler, thank you for the quick reply. Figuring out how to specify the timeout was very helpful. I did run a few tests and will share my results if it helps you or others.

MacOS 12.0.1, Swift 5.5.1 I am mostly uploading jpg files that range for 500k-5MB. The initial reason I had any issue is that I changed locations and have been testing with an average upload speed (speedtest.net) of 2.6Mbps. This led to many files that timeout. Simply changing the timeout time has fixed the issue of files failing to upload. But the experience has led me to explore what throughput I am actually getting.

I won’t say these are precise or scientific measurements, but I offer them here for discussion:

  1. Using a 3.6M jpg file.
  2. Uploaded 3 times using the S3FileTransferManager.copy function
  3. Uploaded 3 times using the S3 web interface using Safari
  4. Performed S3FileTransferManagerTests.testMultipartUploadDownload()

== Spectrum Internet: Upload 2.6Mbps 2. SotoS3FileTransfer upload average = 21 seconds (~1.4Mbps) 3. Safari upload average = 12 seconds (~2.4Mbps) 4. Test Case '-[S3FileTransferManagerTests testMultipartUploadDownload]' passed (75.112 seconds).

== Tethered T-Mobile phone: Upload 1.46Mbps (iPhone7, 1 bar LTE), clearly a poor connection and the results were erratic, but in general the Soto upload time was 50% longer than Safari. 2. SotoS3FileTransfer upload average = 32 seconds 3. Safari upload average = 22 seconds 4. Test Case '-[S3FileTransferManagerTests testMultipartUploadDownload]' passed (123.098 seconds).

geoffhufford avatar Dec 03 '21 20:12 geoffhufford

Multipart upload with Soto v7 uploads multiple parts concurrently now and run a lot faster

adam-fowler avatar Jul 11 '24 16:07 adam-fowler