s3proxy
s3proxy copied to clipboard
Feature/overlay middleware
This adds a new Middleware that implements writes using a local BlobStore and passes reads through to an upstream BlobStore using some simple logic:
- Read requests are returned from the local BlobStore if the key exists, otherwise from the upstream bucket.
- Requests that list container contents are merged between the local and upstream BlobStore
- Writes are written to the local BlobStore
- Delete's are implemented by creating a file in the local BlobStore which indicates the key should be removed from results.
Notes:
- I am not a Java Developer (at least, not in the most recent decade) and this code might not follow all, or many, best practices. Particularly any features used that weren't a part of Java 7.
- This is not production-ready. It was designed for development and testing purposes only. A non-exhaustive list of things that would be needed to be Production Ready are:
- Use a more robust mechanism to generate "Not Found" errors other than trying to load a hopefully non-existent file.
- The local BlobStore might need a mechanism to limit the amount of space it can consume, and how to handle the case where that space is exhausted.
This addresses a common use-case where developers need to test or replicate issues against production storage without actually writing anything back.
This seems like a good feature -- sorry I haven't had time to look at it. Could you squash the commits and resolve the conflicts before I start reviewing it?
@gaul - Good to go