waterbutler
waterbutler copied to clipboard
CenterForOpenScience
[SVCS-548] Chunked Uploads for CloudFiles
Note (Added by Longze)
This PR contains code from: https://github.com/CenterForOpenScience/waterbutler/pull/283.
Ticket
https://openscience.atlassian.net/browse/SVCS-548
Purpose
This PR allows one to upload files larger than 5GBs
Changes
Adds a new method to clould files provider with tests and raises max body limit to arbitrarily high 1 Terra byte.
Side effects
None that I know of.
QA Notes
It will take a long time to upload a > 5 GB files, have something else to do while you wait.
Deployment Notes
None that I know of.
Coverage increased (+0.3%) to 89.511% when pulling a13d088935696c86b874c0411227e05cd26824e1 on Johnetordoff:cloudfiles-large-uploads into cc68aca3fd9dc281aa0d1037ddad95d8eb75a174 on CenterForOpenScience:develop.
Coverage increased (+0.3%) to 89.429% when pulling a75e70fb392db135529095645f4847550dff0154 on Johnetordoff:cloudfiles-large-uploads into ba5573122292d5baa4d62635cab773278e2ee876 on CenterForOpenScience:develop.
Coverage increased (+0.3%) to 89.401% when pulling 71ddc861554451f031a3a286306f49f7aaa760e2 on Johnetordoff:cloudfiles-large-uploads into ba5573122292d5baa4d62635cab773278e2ee876 on CenterForOpenScience:develop.
Coverage increased (+0.3%) to 89.401% when pulling 2bfa324e28a92bfd59489f5d4a736a6487b83109 on Johnetordoff:cloudfiles-large-uploads into ba5573122292d5baa4d62635cab773278e2ee876 on CenterForOpenScience:develop.
Coverage increased (+0.3%) to 89.386% when pulling 680aa72867b329f93fc82f14ecb8edb743901638 on Johnetordoff:cloudfiles-large-uploads into 473191c78c36b6ee63d4609d3952a317ee4ab63e on CenterForOpenScience:develop.
Coverage increased (+0.3%) to 89.386% when pulling 680aa72867b329f93fc82f14ecb8edb743901638 on Johnetordoff:cloudfiles-large-uploads into 473191c78c36b6ee63d4609d3952a317ee4ab63e on CenterForOpenScience:develop.
@icereval thanks. Just fyi that this PR contains code from https://github.com/CenterForOpenScience/waterbutler/pull/283 which adds Cloudfiles as a provider. You might want to take a look at that one as well.
Coverage increased (+0.2%) to 90.191% when pulling 03587f9c54c24fc8efdd7fa491e476aaa0795025 on Johnetordoff:cloudfiles-large-uploads into 26bf2093c15af333e634f14372709e7bf014ccb4 on CenterForOpenScience:develop.
Coverage increased (+0.2%) to 90.191% when pulling 02f4cac362696c1a377ff9bb7fe5bda3b3d4d3a9 on Johnetordoff:cloudfiles-large-uploads into 26bf2093c15af333e634f14372709e7bf014ccb4 on CenterForOpenScience:develop.