mt-aws-glacier icon indicating copy to clipboard operation
mt-aws-glacier copied to clipboard

Ranged Archive Retrieval

Open jeremyd2019 opened this issue 11 years ago • 2 comments

I was just looking at the docs for retrieval (Initiate Job) and download (Get Job Output). And just as I was thinking I'd have to write some custom code for this, you added support for multi-segment download.

The other thing I was thinking I'd have to write some custom code for was the "Ranged Archive Retrieval" feature (documented in http://docs.aws.amazon.com/amazonglacier/latest/dev/api-initiate-job-post.html, see the RetrievalByteRange parameter). Since I don't see an existing issue for this, nor do I see it in your readme for coming-soon or planned-next-version features, I thought I'd try out this issue system and bring it to your attention.

Keep up the good work!

jeremyd2019 avatar Jun 01 '13 21:06 jeremyd2019

This is important because it seems to be the controlling factor of retrieval costs, after reading this thread: https://forums.aws.amazon.com/message.jspa?messageID=374065#374065

jeremyd2019 avatar Jun 01 '13 21:06 jeremyd2019

Hello. Thanks for report.

Yes, I was thinking about range-retrievals. It's indeed important for saving money when restoring data. Problem that I'll need to organize some workflow for this - storage for temporary files.. new data in Journal, new (sane! and easy to understand) command line options. So, this isn't coming soon :(

As workaround you can split your data to archives with same size and retrieve it one-by-one, with max-number-of-files option (ugly, but it works)

Next priority task now is issue number 30 (and of course bugfixes are in priority over enhancements).

btw multi-segment downloads were implemented in emergency, as it seems sometimes it's impossible to download huge files from Amazon https://forums.aws.amazon.com/thread.jspa?threadID=125571&tstart=0

And also I want to mention that coming-soon or planned-next-version sections in documentation now looks outdated to me and should be updated (will update soon and mention this in ChangeLog)

Will leave this ticket open as a placeholder for this enhancement.

vsespb avatar Jun 01 '13 21:06 vsespb