soto-s3-file-transfer
soto-s3-file-transfer copied to clipboard
slow upload speed
Hello
This used to upload fast, now my users and I get 50kb max
did something changed?
I was having timeout issues which I solved using
,timeout: .minutes(5)
Last release was 2 months back. Over what time period have you seen the change?
I just ran some of the tests and am not seeing any slowdown. Try running testMultipartUploadDownload
to see what you get.
Also what platform are you running this on? I guess also are you uploading loads of small files, or a smaller number of large files
I am also exploring slow uploads. How do I set timeout?
Hi @geoffhufford ,
I have a few questions that might help me diagnose your issue
- what platform are you running on
- what version of Swift are you running?
- Are you seeing slowdown when uploading lots of small files, or fewer large files. Are many of your files larger than 5MB?
Can you run the test I mention above to see if you can replicate your issue within that test?
S3.init
has a parameter timeout
. You can also use the with
function which will take an existing S3
object and set the timeout for it
let newS3 = s3.with(timeout: .seconds(120))
@adam-fowler, thank you for the quick reply. Figuring out how to specify the timeout was very helpful. I did run a few tests and will share my results if it helps you or others.
MacOS 12.0.1, Swift 5.5.1 I am mostly uploading jpg files that range for 500k-5MB. The initial reason I had any issue is that I changed locations and have been testing with an average upload speed (speedtest.net) of 2.6Mbps. This led to many files that timeout. Simply changing the timeout time has fixed the issue of files failing to upload. But the experience has led me to explore what throughput I am actually getting.
I won’t say these are precise or scientific measurements, but I offer them here for discussion:
- Using a 3.6M jpg file.
- Uploaded 3 times using the S3FileTransferManager.copy function
- Uploaded 3 times using the S3 web interface using Safari
- Performed S3FileTransferManagerTests.testMultipartUploadDownload()
== Spectrum Internet: Upload 2.6Mbps 2. SotoS3FileTransfer upload average = 21 seconds (~1.4Mbps) 3. Safari upload average = 12 seconds (~2.4Mbps) 4. Test Case '-[S3FileTransferManagerTests testMultipartUploadDownload]' passed (75.112 seconds).
== Tethered T-Mobile phone: Upload 1.46Mbps (iPhone7, 1 bar LTE), clearly a poor connection and the results were erratic, but in general the Soto upload time was 50% longer than Safari. 2. SotoS3FileTransfer upload average = 32 seconds 3. Safari upload average = 22 seconds 4. Test Case '-[S3FileTransferManagerTests testMultipartUploadDownload]' passed (123.098 seconds).
Multipart upload with Soto v7 uploads multiple parts concurrently now and run a lot faster