Blob: Azurite does not replicate API size limit

[theobisproject]

trafficstars

Which service(blob, file, queue, table) does this issue concern?

Blob

Which version of the Azurite was used?

3.29.0

Where do you get Azurite? (npm, DockerHub, NuGet, Visual Studio Code Extension)

mcr.microsoft.com/azure-storage/azurite

What's the Node.js version?

v14.21.3 (inside the container)

What problem was encountered?

Uploading a 10GB file via the BlockBlobClient from the Java SDK is possible in Azurite but not against the real service. There the following error is returned.

Code: RequestBodyTooLarge
Message: The request body is too large and exceeds the maximum permissible limit.
RequestId: <ID>
MaxLimit: 5242880000

Steps to reproduce the issue?

Example code using version 1.2.14 of the azure-sdk-bom

final Path testFile = Paths.get("10GB.dat");
final BinaryData data = BinaryData.fromFile(testFile, (int) FileUtils.ONE_MB);
blobClient.upload(data, false);

Might be related to #1319 where it was only fixed for table

Have you found a mitigation/solution?

Only testing against the real service.

[jrahman]

@blueww @EmmaZhu we've hit this issue as well. It's a bit difficult when Azurite doesn't accurately reflect the actual Azure Blob Storage limits. In our case, we attempted to write 16MB, which Azurite accepted, but actual Azure Storage rejects writes over 4MB. Let's address this soon, the check here is fairly simple, just need to add more validation on the entrypoints to the various APIs here.

[blueww]

@jrahman

Thanks for ping this issue!

Would you please share Azurite debug log for the unexpected success request, so we can be more sure about the root cause of the issue?

From rest API doc, we see:

The block must be less than or equal to 4,000 mebibytes (MiB) in size

The fix might be check the content length before this line of code , to make sure it's <= 4,000 Mib. Else return an error aligned with server error (status code, error code, error message).

Azurite welcome contribution! It would be great if you could raise a PR to fix it!