azure-sdk-for-java icon indicating copy to clipboard operation
azure-sdk-for-java copied to clipboard

[BUG] Imposible to verify existance of a blob container with container-level Shared Access Signature

Open mehdiben7 opened this issue 7 months ago • 4 comments

Describe the bug When connection to an Azure Blob Storage container using a Shared Access Signature created on the container itself, a call to BlobContainerClient.existsWithResponse(Duration, Context) will fail due to non sufficient permission.

Exception or Stack Trace

ERROR when callin .exists()
[main] ERROR com.azure.storage.blob.BlobContainerClient - If you are using a StorageSharedKeyCredential, and the server returned an error message that says 'Signature did not match', you can compare the string to sign with the one generated by the SDK. To log the string to sign, pass in the context key value pair 'Azure-Storage-Log-String-To-Sign': true to the appropriate method call.If you are using a SAS token, and the server returned an error message that says 'Signature did not match', you can compare the string to sign with the one generated by the SDK. To log the string to sign, pass in the context key value pair 'Azure-Storage-Log-String-To-Sign': true to the appropriate generateSas method call.Please remember to disable 'Azure-Storage-Log-String-To-Sign' before going to production as this string can potentially contain PII.If you are using a StorageSharedKeyCredential, and the server returned an error message that says 'Signature did not match', you can compare the string to sign with the one generated by the SDK. To log the string to sign, pass in the context key value pair 'Azure-Storage-Log-String-To-Sign': true to the appropriate method call.If you are using a SAS token, and the server returned an error message that says 'Signature did not match', you can compare the string to sign with the one generated by the SDK. To log the string to sign, pass in the context key value pair 'Azure-Storage-Log-String-To-Sign': true to the appropriate generateSas method call.Please remember to disable 'Azure-Storage-Log-String-To-Sign' before going to production as this string can potentially contain PII.Status code 403, "<?xml version="1.0" encoding="utf-8"?><Error><Code>AuthorizationFailure</Code><Message>This request is not authorized to perform this operation.RequestId:0908e7a9-301e-0055-5aa7-dcada2000000Time:2025-06-13T21:06:19.4965835Z</Message></Error>"
Exception in thread "main" com.azure.storage.blob.models.BlobStorageException: If you are using a StorageSharedKeyCredential, and the server returned an error message that says 'Signature did not match', you can compare the string to sign with the one generated by the SDK. To log the string to sign, pass in the context key value pair 'Azure-Storage-Log-String-To-Sign': true to the appropriate method call.
If you are using a SAS token, and the server returned an error message that says 'Signature did not match', you can compare the string to sign with the one generated by the SDK. To log the string to sign, pass in the context key value pair 'Azure-Storage-Log-String-To-Sign': true to the appropriate generateSas method call.
Please remember to disable 'Azure-Storage-Log-String-To-Sign' before going to production as this string can potentially contain PII.
If you are using a StorageSharedKeyCredential, and the server returned an error message that says 'Signature did not match', you can compare the string to sign with the one generated by the SDK. To log the string to sign, pass in the context key value pair 'Azure-Storage-Log-String-To-Sign': true to the appropriate method call.
If you are using a SAS token, and the server returned an error message that says 'Signature did not match', you can compare the string to sign with the one generated by the SDK. To log the string to sign, pass in the context key value pair 'Azure-Storage-Log-String-To-Sign': true to the appropriate generateSas method call.
Please remember to disable 'Azure-Storage-Log-String-To-Sign' before going to production as this string can potentially contain PII.
Status code 403, "<?xml version="1.0" encoding="utf-8"?><Error><Code>AuthorizationFailure</Code><Message>This request is not authorized to perform this operation.

To Reproduce

  1. Setup an Azure Blob Storage account
  2. Setup a container
  3. Create a valid Shared Access Signature on said container (not on the whole account)
  4. Use it to instantiate a new BlobContainerClient
  5. Call .exists() or .existsWithResponse(Duration, Context) which is called by the first method anyway

Code Snippet

import com.azure.storage.blob.BlobContainerClient;
import com.azure.storage.blob.BlobContainerClientBuilder;

public class Main {
    static final String SAS_CONNECTION_STRING = "DefaultEndpointsProtocol=https;AccountName=sfilerdatastorage;EndpointSuffix=core.windows.net;SharedAccessSignature=sp=racwdl&st=2025-06-09T20:03:04Z&se=2029-07-12T04:03:04Z&spr=https&sv=2024-11-04&sr=c&sig=<real sas here>";
    static final String CONTAINER_NAME = "sfiler-test-sas";

    static final String SAS_URL = "https://sfilerdatastorage.blob.core.windows.net/sfiler-test-sas?sp=racwdl&st=2025-06-09T20:03:04Z&se=2029-07-12T04:03:04Z&spr=https&sv=2024-11-04&sr=<real sas here>";
    
    public static void main(String[] args) {
        BlobContainerClient client1 = new BlobContainerClientBuilder()
                .connectionString(SAS_CONNECTION_STRING)
                .containerName(CONTAINER_NAME)
                .buildClient();

        BlobContainerClient client2 = new BlobContainerClientBuilder()
                .endpoint(SAS_URL)
                .buildClient();

        // list both containers
        System.out.println("Listing blobs in container using connection string:");
        client1.listBlobs().forEach(blobItem -> {
            System.out.println("Blob name: " + blobItem.getName());
        });
        System.out.println("Listing blobs in container using SAS URL:");
        client2.listBlobs().forEach(blobItem -> {
            System.out.println("Blob name: " + blobItem.getName());
        });

        // test container .exists
        System.out.println("Container exists using connection string: " + client1.exists());
        System.out.println("Container exists using SAS URL: " + client2.exists());
    }
}

Here both call to .exists() crash yet listing files, creating and deleting them etc, works like a charm. This is not the real name of my storage account and I have obfuscated the SAS token but I ensure you it has been copied correctly, as I have checked dozens of times.

Expected behavior Either the Azure Blob Storage should not return a Forbidden HTTP status code OR they should be another way to verify that the container exists before attempting to use it

Setup (please complete the following information):

  • OS: Windows 11 Pro
  • IDE: IntelliJ IDEA 2025.1.1.1 (Ultimate edition)
  • Library/Libraries: com.azure:azure-storage-blob:12.30.0, com.azure:azure-identity:1.16.1, com.azure:azure-core:1.55.3
  • Java version: 21
  • App Server/Environment: not applicable
  • Frameworks: not applicable

Additional context I am developping a SaaS solution that uses Azure Blob Storage as one of the option store files. There was found a need for us to be able to provide support for using Shared Access Signature instead of an account key.

The "test access" function of our software did not work, yet it could store files and do normal operations properly. The problem also rises because we attempt to check if the storage actually exists before making operations on the container.

After investigation, it seems as if the problem is with the .exists() method. It works perfectly

  • using an account key
  • using an account level SAS

but not when using a container level SAS

Information Checklist Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report

  • [x] Bug Description Added
  • [x] Repro Steps Added
  • [x] Setup information Added

mehdiben7 avatar Jun 13 '25 21:06 mehdiben7

@gunjansingh-msft @ibrandes @kyleknap @seanmcc-msft

github-actions[bot] avatar Jun 13 '25 21:06 github-actions[bot]

Thank you for your feedback. Tagging and routing to the team member best able to assist.

github-actions[bot] avatar Jun 13 '25 21:06 github-actions[bot]

If I may also add some context: at first one of the solution my team and I found was to create and immediately delete a small file, which works perfectly fine, but it is kind of gross. One of our client does thousands of requests per second and it would therefore not be a valid solution

mehdiben7 avatar Jun 13 '25 21:06 mehdiben7

If I may also add some context: at first one of the solution my team and I found was to create and immediately delete a small file, which works perfectly fine, but it is kind of gross. One of our client does thousands of requests per second and it would therefore not be a valid solution

I also want to add that we cannot list files as a non working connection will yield an empty array, the same as the connection was working but had no files

mehdiben7 avatar Jun 16 '25 13:06 mehdiben7

up?

mehdiben7 avatar Jun 22 '25 22:06 mehdiben7

?

mehdiben7 avatar Jul 01 '25 16:07 mehdiben7

Hi @mehdiben7,

Thanks for reaching out! The behavior you're seeing is actually expected. As outlined in the documentation, a container-level SAS only grants access to the content and metadata of blobs within the container, as well as the ability to list blobs. The container.exists() method internally calls Get Container Properties, which requires metadata/property read permissions not included in a container-level SAS.

Is there a specific reason your team is avoiding the use of an account-level SAS? If it's due to security concerns, it's worth noting that account-level SAS tokens can be tightly scoped using various constraints to limit their exposure.

ibrandes avatar Jul 01 '25 19:07 ibrandes

Hi @mehdiben7,

Thanks for reaching out! The behavior you're seeing is actually expected. As outlined in the documentation, a container-level SAS only grants access to the content and metadata of blobs within the container, as well as the ability to list blobs. The container.exists() method internally calls Get Container Properties, which requires metadata/property read permissions not included in a container-level SAS.

Is there a specific reason your team is avoiding the use of an account-level SAS? If it's due to security concerns, it's worth noting that account-level SAS tokens can be tightly scoped using various constraints to limit their exposure.

Thank you for your answer

The reason why our team wants to implement support for container-level SAS is as follows:

Some of our customers may want to use our solution with multiple containers per storage account. With container-level SAS, it is absolutely impossible for our application to access the other containers. It's an extra level of isolation that some customers might want to use. It bothers us a little that the solution “only” supports SAS at storage account level. We want to be able to support the use of Azure Blob in all possible scenarios.

Let me know if that answers your question and if so if you can think of any proper way to handle this usecase other than the "hack" I talked about earlier

mehdiben7 avatar Jul 03 '25 21:07 mehdiben7

Thank you for your answer

The reason why our team wants to implement support for container-level SAS is as follows:

Some of our customers may want to use our solution with multiple containers per storage account. With container-level SAS, it is absolutely impossible for our application to access the other containers. It's an extra level of isolation that some customers might want to use. It bothers us a little that the solution “only” supports SAS at storage account level. We want to be able to support the use of Azure Blob in all possible scenarios.

Let me know if that answers your question and if so if you can think of any proper way to handle this usecase other than the "hack" I talked about earlier

Thanks for the detailed explanation — that makes sense, and we understand the need for container-level isolation in multi-container scenarios.

You're free to use whatever workaround fits your needs for now. That said, proper support for container-level SAS would require changes on the storage service side, so it's not something we can implement independently in the SDK. Please consider submitting this feedback to the Storage service team via upvoting or creating a new idea at https://aka.ms/AzStorageFeedback.

This will help the service team gauge interest and prioritize new features. If the feature becomes available in the service, it will be promptly added to the SDK.

ibrandes avatar Jul 08 '25 17:07 ibrandes