s3proxy
s3proxy copied to clipboard
Middleware - sharding to multiple backends
Is this currently possible? I see I can shard across virtual buckets which is nice but it's not clear on if it would allow sharding across multiple backends (in this case it would be identical backends using the file system). That feature would be great.
Although now that I think about it - technically it would be the same backend type (file system) but if i could at least choose different backend paths for the sharding that could work.
This is a reasonable feature request and something the startup Bounce Storage implemented. It is easy to make S3Proxy write to multiple backends, at least when all the putBlob requests succeed. However it is more difficult to create policies for reading from a backend. Does it only read from one of them? Does it try to read from one and fail over to the second? What happens when writes fail to only one of the backends?