S3Client on filesystem-based proxy can't delete objects

[InsertPikachuFace]

Calling DeleteObject in my junit integration-test does not work. The corresponding file in my temporary folder still exists and nothing changed in the result of a HeadObject call (no "NoSuchKeyException", no "DeleteMarker", same "LastModified" timestamp, etc). A waiter for object-not-exists just goes on until timeout.

I don't know if this is a problem with my configuration or the S3Proxy, but based on everything I could find this seems like a problem with the filesystem-interaction.

My specs are

• Windows 10
• OpenJDK 23.0.1
• (Maven) software.amazon.awssdk.s3:2.26.12
• (Maven) org.gaul.s3proxy:2.6.0

Here is a basic recreation of my test as a main() method:

public void main() throws Exception {
    File s3ProxyTempDir = Files.createTempDirectory(this.getClass().getSimpleName()).toFile();
    // s3ProxyTempDir.deleteOnExit();
    System.out.println("s3ProxyTempDir: " + s3ProxyTempDir.getAbsolutePath());
    Region region = Region.EU_CENTRAL_1;
    Properties properties = new Properties();
    properties.setProperty("jclouds.filesystem.basedir", s3ProxyTempDir.getPath());
    properties.setProperty("jclouds.regions", region.id());

    BlobStoreContext blobStoreContext = ContextBuilder
        .newBuilder("filesystem")
        .credentials("identity", "credential")
        .overrides(properties)
        .build(BlobStoreContext.class);

    URI endpoint = URI.create("http://127.0.0.1:8080");
    S3Proxy s3Proxy = S3Proxy.builder()
        .awsAuthentication(AuthenticationType.NONE, null, null)
        .blobStore(blobStoreContext.getBlobStore())
        .endpoint(endpoint)
        .build();

    s3Proxy.start();
    while (!s3Proxy.getState().equals(AbstractLifeCycle.STARTED)) {
        Thread.sleep(100);
    }

    S3Client s3Client = S3Client.builder()
        .region(region)
        .endpointOverride(endpoint)
        .credentialsProvider(StaticCredentialsProvider.create(AwsBasicCredentials.create("access", "secret")))
        .build();
    System.out.println("S3Proxy started & client initialised");

    String bucketName = "test";
    String fileKey = "dummy.txt";
    s3Client.createBucket(b -> b.bucket(bucketName));
    System.out.println("waiting for bucket creation");
    s3Client.waiter().waitUntilBucketExists(b -> b.bucket(bucketName));
    System.out.println("created bucket");

    s3Client.putObject(b -> b.bucket(bucketName).key(fileKey), RequestBody.fromString("12345"));
    System.out.println("waiting for file creation");
    s3Client.waiter().waitUntilObjectExists(b -> b.bucket(bucketName).key(fileKey));
    System.out.println("put object");

    s3Client.deleteObject(b -> b.bucket(bucketName).key(fileKey));
    System.out.println("waiting for object deletion");
    s3Client.waiter().waitUntilObjectNotExists(b -> b.bucket(bucketName).key(fileKey));
    System.out.println("deleted object"); // is not executed
}

[pvrhijn]

I have the same issue

[gaul]

Could you replicate this in the S3Proxy unit tests? AwsSdkTest.testBlobRemove appears to do the same thing as your example so I wonder if there is some difference in the S3 clients. If you want a true same test you could add this to AwsSdkTest2 which uses the same SDK.