Andrew Gaul

S3Proxy could redirect clients to the storage backend via HTTP 307 with a signed URL via a middleware. This would allow S3Proxy to sit out of path of the data,...

Allow configuration via command-line

1

This would simplify many of the examples, especially for the common use case of the filesystem provider with no authentication.

S3Proxy could reduce its thread usage by asynchronously streaming data instead of synchronously copying it. `BlobStore.getBlob` provides an `InputStream` which allows this but `BlobStore.putBlob` does not expose an `OutputStream` to...

Native multipart copy

1

Presently S3Proxy emulates multipart copy with a range `getBlob` request followed by `uploadPart`. Instead jclouds should offer native support. Note that this will need the Azure part size workaround as...

V4 signature payload signing

1

Presently S3Proxy ignores chunk signatures in `ChunkedInputStream`. Reference: https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-streaming.html#sigv4-chunked-body-definition

This would run the unit tests but more importantly run the s3-tests conformance suite. This would allow developers to easily run the same tests as Travis.

Many s3-tests fail due to unreadable characters in headers: ``` ERROR: s3tests.functional.test_headers.test_object_create_bad_expect_unreadable ERROR: s3tests.functional.test_headers.test_object_create_bad_ua_unreadable ERROR: s3tests.functional.test_headers.test_bucket_create_bad_expect_unreadable ERROR: s3tests.functional.test_headers.test_bucket_create_bad_ua_unreadable ERROR: s3tests.functional.test_s3.test_object_set_get_metadata_empty_to_unreadable_prefix ERROR: s3tests.functional.test_s3.test_object_set_get_metadata_empty_to_unreadable_suffix ERROR: s3tests.functional.test_s3.test_object_set_get_metadata_empty_to_unreadable_infix ERROR: s3tests.functional.test_s3.test_object_set_get_metadata_overwrite_to_unreadable_prefix ERROR: s3tests.functional.test_s3.test_object_set_get_metadata_overwrite_to_unreadable_suffix ERROR: s3tests.functional.test_s3.test_object_set_get_metadata_overwrite_to_unreadable_infix...

Object versioning

2

S3Proxy does not currently support object versioning, triggering many s3-tests failures: ``` ERROR: s3tests.functional.test_s3.test_versioning_bucket_create_suspend ERROR: s3tests.functional.test_s3.test_versioning_obj_create_read_remove ERROR: s3tests.functional.test_s3.test_versioning_obj_create_read_remove_head ERROR: s3tests.functional.test_s3.test_versioning_obj_suspend_versions ERROR: s3tests.functional.test_s3.test_versioning_obj_suspend_versions_simple ERROR: s3tests.functional.test_s3.test_versioning_obj_create_versions_remove_all ERROR: s3tests.functional.test_s3.test_versioning_obj_create_overwrite_multipart ERROR: s3tests.functional.test_s3.test_versioning_obj_list_marker ERROR: s3tests.functional.test_s3.test_versioning_copy_obj_version...

jclouds demangles blob names to emulate directories which are not useful to S3Proxy. @kahing suggests working around this with: ``` java try { Field f = BlobStoreConstants.class.getDeclaredField("DIRECTORY_SUFFIXES"); f.setAccessible(true); Field modifiersField...