s3fs-nio icon indicating copy to clipboard operation
s3fs-nio copied to clipboard

Implement support for MinIO

Open carlspring opened this issue 4 years ago • 2 comments

Task Description

We need to investigate what would be required in order to add support for MinIO.

Tasks

The following tasks will need to be carried out:

  • [ ] Investigate what will need to be done.
  • [ ] Implement the necessary changes.
  • [ ] ~~Create a MinIOITTestSuite based on the AmazonS3ITSuite one.~~ (check #183)
    • [ ] Implement a JUnit extension that starts the MinIO docker image using testcontainers before all tests in the suite (and then stops it at the end). (check #60)
  • [ ] Add test cases.
  • [ ] Illustrate how to use this in the documentation.

Task Relationships

This task:

  • Relates to: #60

Useful Links

Help

  • Our chat channel
  • Points of contact:
    • @carlspring
    • @steve-todorov
    • @ptirador

carlspring avatar Nov 08 '20 02:11 carlspring

Hi,

i investigated today and it seems there is no need for any significant changes in the codebase. I was able to configure MinIO without any problems as a drop in replacement.

I used the following Test to verify the behavior:

        final String accessKey = "AKIAIOSFODNN7EXAMPLE";
        final String secretKey = "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY";
        final URI server = URI.create("s3://localhost:9000");
        final Map<String, String> env = new HashMap<>();
        env.put(S3Factory.ACCESS_KEY, accessKey);
        env.put(S3Factory.SECRET_KEY, secretKey);
        env.put(S3Factory.PROTOCOL, "http");
        try (final FileSystem fs = FileSystems.newFileSystem(server, env, Thread.currentThread().getContextClassLoader())) {
            final Path path = fs.getPath("/meow/XYZ.txt");
            System.out.println(path);
            Files.write(path, "Hello World\nLine 1\n\nline 2\n".getBytes(StandardCharsets.UTF_8));
            System.out.println(Files.readAllLines(path));
        }

While running the MinIO Container with the following docker command (taken from here https://docs.min.io/docs/minio-docker-quickstart-guide )

docker run -p 9000:9000 \
  -e "MINIO_ACCESS_KEY=AKIAIOSFODNN7EXAMPLE" \
  -e "MINIO_SECRET_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY" \
  minio/minio server /data

I'll go work on the Test Container issue now and add some tests for this Issue as well.

BjoernAkAManf avatar Nov 26 '20 14:11 BjoernAkAManf

Awesome! Thanks for these great findings! :)

carlspring avatar Nov 26 '20 19:11 carlspring