s3proxy
s3proxy copied to clipboard
Add middleware to mirror write operations to two storage backends
S3Proxy can use jclouds ForwardingBlobStore
to mirror write operations to two storage backends. Failure to write to either store should fail the client request. Note that this will leave the store in an inconsistent state that some external service needs to reconcile. Servicing read operations should try the first backend first and fail over to the second on errors.
@dleute Related to your request in #93.
Thanks! Unfortunately my project priorities have shifted to new projects. I would love to see how this could be implemented though. What I was trying to do was create minimal long term maintenance. Using a ForwardingBlobStore still isn't clear to me if it does that. It would seem like I would have to embed additional config and new code.
This is a really cool idea.
Just a thought - if you wanted, you could read from both (all?) configured storage services when reads are processed and ensure the checksums match before responding with a success to the client. This sort of 'read-time checking' is similar to what other eventually consistent databases do.
Need to fix JCLOUDS-769 to address this.