Gaufrette icon indicating copy to clipboard operation
Gaufrette copied to clipboard

AwsS3 Multipart Uploads

Open MonsieurLanza opened this issue 7 years ago • 10 comments

This is a modification of AwsS3 adapter to allow multipart uploads when files are bigger than a certain size.

Size limits / part size are passed as options to the adapter and set to defaults, respectively 5Go and 5Mo. 5Go being the maximum size AWS S3 accepts in a single upload, and 5Mo the minimum part size.

Limitations : won't work with string content, you have to pass a resource handle otherwise write() will return false. We may use InMemoryStream to bypass this limitation ?

PHP7.1 fails on docker image, not sure why. May need some help on this.

MonsieurLanza avatar Jun 30 '17 15:06 MonsieurLanza

Hope I did not forget anything :)

MonsieurLanza avatar Jul 02 '17 13:07 MonsieurLanza

Can you rebase on master (to consider the merge of #514 ), and update your test please (note the changes)?

nicolasmure avatar Jul 03 '17 07:07 nicolasmure

Done.

MonsieurLanza avatar Jul 04 '17 07:07 MonsieurLanza

Fix : Use Amazon's size limit for string upload and option specific limit for resource handle to allow 5Go upload no matter what. Added a test case to cover that.

MonsieurLanza avatar Jul 04 '17 12:07 MonsieurLanza

Hello, some news here ? We would like this PR to get rid of our custom repo. :)

MonsieurLanza avatar Jul 19 '17 09:07 MonsieurLanza

Hi, are there plans to merge this? I'm having troubles with this.

Thanks!

amcastror avatar Mar 01 '18 18:03 amcastror

Hi guys, Is this possible to merge this pull request ? Need it!! Thank you

JoydS avatar Mar 28 '18 14:03 JoydS

@NiR- we would really appreciate your review on this since @nicolasmure approved it some time ago :)

thank you

jfouca avatar Nov 06 '19 16:11 jfouca

Hello, any idea when this would be merged? Love the concept, would release some workarounds form my project.

amodliszewski avatar Jan 17 '20 07:01 amodliszewski

Btw. Version 0.10.0 will support AsyncAws and that switches to multipart uploads automatically.

It will not have a memory foot print that is related to the part size.

Nyholm avatar Oct 05 '20 19:10 Nyholm