micronaut-core icon indicating copy to clipboard operation
micronaut-core copied to clipboard

File upload is limited to 1GB when using MultipartBody

Open loicmathieu opened this issue 1 year ago • 3 comments

Expected Behavior

We ca upload files of anysize when using MultipartBody

Actual Behaviour

I have an endpoint that can handle multipart request with multiple parts of unknown names, some are files, some are not. The only way to deal with such endpoint is to bind the body to @Body MultiparBody body then doing something like that:

Flux.from(body)
            .subscribeOn(Schedulers.boundedElastic())
            .map(part -> {
                if (part instanceof CompletedFileUpload fileUpload) {
                    File tempFile = File.createTempFile(fileUpload.getFilename() + "_", ".upl");
                    try (var inputStream = fileUpload.getInputStream();
                        var outputStream = new FileOutputStream(tempFile)) {
                        long transferredBytes = inputStream.transferTo(outputStream);
                        System.out.println(fileUpload.getSize()); // 1681355024 => 1.7GB: OK
                        System.out.println(transferredBytes); // 1073708906 => 1GB !!!
                    }
                    return new AbstractMap.SimpleEntry<>(
                        fileUpload.getFilename(),
                        tempFile.getName()
                    );

                } else {
                    return new AbstractMap.SimpleEntry<>(
                        input.getName(),
                        new String(input.getBytes())
                    );
                }
            })
            .collectMap(AbstractMap.SimpleEntry::getKey, AbstractMap.SimpleEntry::getValue)
            .block();

However, when I create the file, it only store 1GB whereas my file in the part is approx 1.7GB. When I call fileUpload.getSize() it correctly reports the correct file size (1.7GB in my case) but the generated file is only 1GB long, in fact it seems to be the underlying InputStream that limit to 1BG.

Steps To Reproduce

No response

Environment Information

No response

Example Application

No response

Version

4.3.4

loicmathieu avatar Mar 29 '24 15:03 loicmathieu

probably a bug in MicronautHttpData

yawkat avatar Apr 02 '24 09:04 yawkat

I have a reproducer, it happens only with micronaut.server.multipart.disk=true or micronaut.server.multipart.mixed=true, with the default it works but the request is very slow to answer (multiple minutes whereas it took a few seconds with disk or mixed).

This is the reproducer project: issue-multipart-body.zip

It can be reproduced using the following cURL:

curl -XPOST -F "[email protected];filename=file1" -F "string1=value1" http://localhost:8080/upload/file1

loicmathieu avatar Apr 26 '24 15:04 loicmathieu

Just checked Micronaut Platform 4.4.1 and the issue still exists.

loicmathieu avatar Apr 26 '24 15:04 loicmathieu

Thanks for the report!

yawkat avatar May 13 '24 17:05 yawkat