aws-sdk-java icon indicating copy to clipboard operation
aws-sdk-java copied to clipboard

Using `BufferedInputStream` causes an upload error

Open JavaLionLi opened this issue 3 years ago • 9 comments

Describe the bug

Upload files to receive using MultipartFile MultipartFile.getInputStream return BufferedInputStream Using BufferedInputStream causes an upload error

Expected Behavior

use bytes upload success

Current Behavior

image image

Reproduction Steps

image

image

Possible Solution

No response

Additional Information/Context

No response

AWS Java SDK version used

1.12.341

JDK version used

1.8.0_202

Operating System and version

centos7.6

JavaLionLi avatar Nov 13 '22 14:11 JavaLionLi

Hi @JavaLionLi can you share a repro code in text format, not screenshot? We ask for repro code for two main reasons: to better understand what the application is trying to do, and to be able to simply copy+paste the code and run it in our side to reproduce the error.

debora-ito avatar Nov 15 '22 22:11 debora-ito

Hi @JavaLionLi can you share a repro code in text format, not screenshot? We ask for repro code for two main reasons: to better understand what the application is trying to do, and to be able to simply copy+paste the code and run it in our side to reproduce the error.

Simply put, PutObjectRequest use BufferedInputStream error

JavaLionLi avatar Nov 16 '22 02:11 JavaLionLi

Hi @JavaLionLi can you share a repro code in text format, not screenshot? We ask for repro code for two main reasons: to better understand what the application is trying to do, and to be able to simply copy+paste the code and run it in our side to reproduce the error.

        try {
            // The file must be larger than 200kb
            BufferedInputStream inputStream = new BufferedInputStream(new FileInputStream("D:\test.txt"));
            PutObjectRequest putObjectRequest = new PutObjectRequest("test", path, inputStream, new ObjectMetadata());
            client.putObject(putObjectRequest);
        } catch (Exception e) {
            e.printStackTrace();
        }

JavaLionLi avatar Nov 16 '22 02:11 JavaLionLi

Can you share a code sample of how the s3 client is being created?

debora-ito avatar Nov 18 '22 00:11 debora-ito

Can you share a code sample of how the s3 client is being created?

        AwsClientBuilder.EndpointConfiguration endpointConfig =
            new AwsClientBuilder.EndpointConfiguration("http://127.0.0.1:9000", "");
        AWSCredentials credentials = new BasicAWSCredentials("admin", "123456");
        AWSCredentialsProvider credentialsProvider = new AWSStaticCredentialsProvider(credentials);
        ClientConfiguration clientConfig = new ClientConfiguration();
        AmazonS3ClientBuilder build = AmazonS3Client.builder()
            .withEndpointConfiguration(endpointConfig)
            .withClientConfiguration(clientConfig)
            .withCredentials(credentialsProvider)
            .disableChunkedEncoding();
        this.client = build.build();

use minio

JavaLionLi avatar Nov 18 '22 06:11 JavaLionLi

I can't reproduce the issue. I used the s3 endpoint though, not minio. Maybe it's a minio issue, do you see the same error when using the s3 endpoint directly?

You mentioned Multipart in the description, but the repro code calls a regular non-multipart PutObject, can you share a repro code for Multipart Upload? And if possible, please enable the DEBUG logs and share here when the error occurs. For instructions on how to enable DEBUG logs check Request/Response Summary Logging here: https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/java-dg-logging.html#sdk-net-logging-request-response

debora-ito avatar Nov 22 '22 00:11 debora-ito

I can't reproduce the issue. I used the s3 endpoint though, not minio. Maybe it's a minio issue, do you see the same error when using the s3 endpoint directly?

You mentioned Multipart in the description, but the repro code calls a regular non-multipart PutObject, can you share a repro code for Multipart Upload? And if possible, please enable the DEBUG logs and share here when the error occurs. For instructions on how to enable DEBUG logs check Request/Response Summary Logging here: https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/java-dg-logging.html#sdk-net-logging-request-response

Nothing to do with minio, Using a BufferedInputStream file larger than 500kb will result in an error, The error was reported before it was uploaded image image BufferedInputStream uses reset to report errors, which can be found on the network

JavaLionLi avatar Nov 22 '22 04:11 JavaLionLi

No one to deal with it?

JavaLionLi avatar Feb 05 '23 05:02 JavaLionLi

I had the same problem in sdk 2.x , So I try use java reflection to get BufferedInputSteam filed in (FileInputStream) and put it . @JavaLionLi code like:

InputStream inputStream = file.getInputStream();
Field in = FilterInputStream.class.getDeclaredField("in");
in.setAccessible(true);
InputStream inNew = (InputStream)in.get(inputStream);
PutObjectResponse putObjectResponse = s3Client.putObject(putObjectRequest, RequestBody.fromInputStream(inNew, size));

hqbhoho avatar Oct 15 '23 06:10 hqbhoho

@JavaLionLi

I apologize but I didn't have the chance to troubleshoot this further. We are closing old v1 issues before going into Maintenance Mode, so I recommend you test it using v2 and if this issue still persists please open a new issue in the v2 repo.

Reference:

  • Announcing end-of-support for AWS SDK for Java v1.x effective December 31, 2025 - blog post

debora-ito avatar Jul 10 '24 19:07 debora-ito

This issue is now closed.

Comments on closed issues are hard for our team to see. If you need more assistance, please open a new issue that references this one.

github-actions[bot] avatar Jul 10 '24 19:07 github-actions[bot]