azure-sdk-for-java icon indicating copy to clipboard operation
azure-sdk-for-java copied to clipboard

[BUG] BlobContainerClientBuilder incorrectly rebuilds URL with nested container name

Open evilmilo opened this issue 3 years ago • 4 comments

Describe the bug We trying to use v12 Java SDK to upload a blob using a SAS URL that contains nested container name, but the URL that is actually sent is missing the full container name, only sending the first part

Exception or Stack Trace <AuthenticationErrorDetail>Signature did not match. String to sign used was...

To Reproduce

final BlobContainerClient blobContainerClient = new BlobContainerClientBuilder()
    .endpoint("https://ourserver.blob.core.windows.net/taxfiles/a098b5e2-482b-4e86-9cc4-499a69dbea7f/bird23.xls?sv=2021-08-06&st=2022-08-22T14...")
    .buildClient();
			
final BlobClient blobClient = blobContainerClient.getBlobClient(fileName);
blobClient.upload(fileStream.getStream(), fileStream.getLength());

using postman, can see that it actually tries to send to:

https://ourserver.blob.core.windows.net/taxfiles/bird23.xls?sv=2021-08-06&st=2022-08-22T14...

Doing similar with v8 SDK DOES work and sends to full proper URL:

final CloudBlockBlob cloudBlockBlob = new CloudBlockBlob(URI.create(connectionDetails.getUrl()));
cloudBlockBlob.upload(fileStream.getStream(), fileStream.getLength());

Expected behavior If BlobContainerClient built from endpoint url, then should use that same URL to upload the blob, not truncate part of the container causing invalid request/signature. Would expect upgrading from v8 to v12 would not break previous simple case to provide URL to upload to.

Setup (please complete the following information):

  • OS: IOS

  • IDE: IntelliJ

  • Library/Libraries: com.azure:azure-core:1.28.0 com.azure:azure-identity:1.4.6 com.azure:messaging-servicebus:7.7.0 com.azure:azure-storage-blob:12.16.1

  • Java version: 17

Information Checklist Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report

  • [x] Bug Description Added
  • [x] Repro Steps Added
  • [x] Setup information Added

evilmilo avatar Aug 22 '22 17:08 evilmilo

Hi, @evilmilo Thank you for opening this issue. Can you please clarify what you mean by "nested container name"? This isn't a standard term used by Storage? In your example, what is the expected container name and what is the file name?

rickle-msft avatar Aug 23 '22 17:08 rickle-msft

So I think finally figured out our issue. We are being given SAS url by a 3rd party, and just want to use that to do upload. They do that with just a PUT request with the URL, but our tech team decided safer to use the azure SDK.

The issue is that if you use v8 or you use PUT, you don't have to understand the URL, you can just use it. To do the same in v12 think we FINALLY figured out need:

BlobClient blobClient = new BlobClientBuilder()
    .endpoint("<your-storage-account-url>" + "/" + "mycontainer" + "/" + "myblob" + "?" + "<your-sasToken>")
    .buildClient();

But this was not clear from following the online docs we had: https://docs.microsoft.com/en-us/azure/storage/blobs/storage-quickstart-blobs-java?tabs=powershell%2Cenvironment-variable-windows

as examples seemed to show we had to get BlobClient from BlobContainerClient, and then assumed that filename was same as blob name.

however since azure doesn't have nested containers (from what I've understood), it turns out that for URL sending to: https://ourserver.blob.core.windows.net/taxfiles/a098b5e2-482b-4e86-9cc4-499a69dbea7f/bird23.xls

the container is: taxfiles the blob name is: a098b5e2-482b-4e86-9cc4-499a69dbea7f/bird23.xls

This understanding of containers and blob names wasn't obvious to us, so both us and 3rd party we were working with couldn't figure out why wasn't working with:

blobContainerClient.getBlobClient("bird23.xls");

as only gave us back the signature error. It was only by using POSTMAN to record responses finally saw that we were losing

a098b5e2-482b-4e86-9cc4-499a69dbea7f

which we had assumed was part of container name.

Possibly better docs or better error message or suggestions on bad signature? Is there an easy way to get azure to log or show the url it is going to actually send to as this was key to figuring out issue, but as I say had to learn how to set up POSTMAN proxy and capture before we made headway on this problem.

evilmilo avatar Aug 24 '22 14:08 evilmilo

@evilmilo Glad to hear it's sorted out! And thank you for the feedback in docs. I can turn this into a FeatureReq to update docs. Would you say updating the quickstart to show alternate methods of getting to a blob client would have been the most helpful for you guys?

Updating the error messaging is tough because when the service comes back with an auth failure, it doesn't give us any information that we can act on. We do have a message that suggest printing out the string to sign, which also would have revealed the difference in the target resource. Alternatively, enabling any of the forms of logging probably would also have printed the target of the outgoing requests.

rickle-msft avatar Aug 24 '22 23:08 rickle-msft

Hi @evilmilo , just following up on this thread. We would like to know what part of the docs you'd suggest we update before we put in a feature request (since updating error message may not be possible). Would you think updating the quickstart to show alternate methods of getting to a blob client would be helpful for you?

Please let us know, thank you!

ibrahimrabab avatar Sep 20 '22 21:09 ibrahimrabab