Dynamic Provisioning fails if using Storage Account from another subscription

[davidrsnd]

What happened: PV fails to be dynamically provisioned for Azure File if Storage Account is from another subscription.

xxx@yyy:~$ k describe pvc testpvc2
Name:          testpvc2
Namespace:     default
StorageClass:  test
Status:        Pending
Volume:
Labels:        <none>
Annotations:   volume.beta.kubernetes.io/storage-provisioner: file.csi.azure.com
Finalizers:    [kubernetes.io/pvc-protection]
Capacity:
Access Modes:
VolumeMode:    Filesystem
Used By:       <none>
Events:
  Type     Reason                Age               From                                                                                               Message
  ----     ------                ----              ----                                                                                               -------
  Normal   Provisioning          6s (x5 over 21s)  file.csi.azure.com_csi-azurefile-controller-754d89466b-7lb4w_da9e66a7-3523-448c-9130-025e6a91b03c  External provisioner is provisioning volume for claim "default/testpvc2"
  Warning  ProvisioningFailed    6s (x5 over 21s)  file.csi.azure.com_csi-azurefile-controller-754d89466b-7lb4w_da9e66a7-3523-448c-9130-025e6a91b03c  failed to provision volume with StorageClass "test": rpc error: code = Internal desc = storage.FileSharesClient#Get: Failure responding to request: StatusCode=403 -- Original Error: autorest/azure: Service returned an error. Status=403 Code="AuthorizationFailed" Message="The client 'c07d8997-6f0e-4808-bede-bb5c1da3200e' with object id 'c07d8997-6f0e-4808-bede-bb5c1da3200e' does not have authorization to perform action 'Microsoft.Storage/storageAccounts/fileServices/shares/read' over scope '/subscriptions/**5e0ca85f-86ed-4334-b634-a115273161d9**/resourceGroups/sandbox-rg-2022-09-05/providers/Microsoft.Storage/storageAccounts/xyz/fileServices/default/shares/pvc-752dc9c8-e5db-40de-b5bd-0ffc365d5252' or the scope is invalid. If access was recently granted, please refresh your credentials."
  Normal   ExternalProvisioning  4s (x3 over 21s)  persistentvolume-controller                                                                        waiting for a volume to be created, either by external provisioner "file.csi.azure.com" or manually created by system administrator

Note that the Subscription ID on the error is diferent then what defined in the SC above 5e0ca85f-86ed-(...) is the SubId of the AKS cluster 246c1103-d260-(...) is the SubID of the alien Storage account

The necessary permissions are granted.

xxx@yyy:~$ k describe sc test
Name:            test
IsDefaultClass:  No
Annotations:     kubectl.kubernetes.io/last-applied-configuration={"allowVolumeExpansion":true,"apiVersion":"storage.k8s.io/v1","kind":"StorageClass","metadata":{"annotations":{},"name":"test"},"mountOptions":["dir_mode=0555","file_mode=0444"],"parameters":{"resourceGroup":"sandbox-rg-2022-09-05","storageAccount":"xyz","subscriptionID":"246c1103-d260-4d45-aa9b-beb707432fc1"},"provisioner":"file.csi.azure.com"}

Provisioner:           file.csi.azure.com
Parameters:            resourceGroup=sandbox-rg-2022-09-05,storageAccount=xyz,subscriptionID=**246c1103-d260-4d45-aa9b-beb707432fc1**
AllowVolumeExpansion:  True
MountOptions:
  dir_mode=0555
  file_mode=0444
ReclaimPolicy:      Delete
VolumeBindingMode:  Immediate
Events:             <none>

What you expected to happen: Dynamic Provisioning using a Storage Account from another subscription should work. Pod , PVC an PV should be successfully created.

How to reproduce it: For Dynamic Provisioning, Create a SC with Driver Parameters resourceGroup, storageAccount and subscriptionID from another subscription. Create PVC and pod based on that SC. Describe PVC which will show the warning in the events.

Anything else we need to know?:

Environment:

• CSI Driver version:

kubectl get po -n kube-system -o yaml | grep mcr | grep azurefile
      image: mcr.microsoft.com/oss/kubernetes-csi/azurefile-csi:v1.21.0
      image: mcr.microsoft.com/oss/kubernetes-csi/azurefile-csi:v1.21.0
      imageID: mcr.microsoft.com/oss/kubernetes-csi/azurefile-csi@sha256:b21ba2841a705f0f9fef5afa9f3f50c5cb11ddb504028626cc507966c52edf30

• Kubernetes version (use kubectl version):

Client Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.3", GitCommit:"816c97ab8cff8a1c72eccca1026f7820e93e0d25", GitTreeState:"clean", BuildDate:"2022-01-25T21:25:17Z", GoVersion:"go1.17.6", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.8", GitCommit:"a35ccc8395e8a5eaa83b7b8c981677893b651cba", GitTreeState:"clean", BuildDate:"2022-07-28T20:52:46Z", GoVersion:"go1.17.11", Compiler:"gc", Platform:"linux/amd64"}

• OS (e.g. from /etc/os-release):

NAME="Ubuntu"
VERSION="20.04.3 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.3 LTS"
VERSION_ID="20.04"

• Kernel (e.g. uname -a): Linux DESKTOP-TP4F0OG 5.10.102.1-microsoft-standard-WSL2 #1 SMP Wed Mar 2 00:30:59 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
• Install tools:
• Azure Managed - CSI driver installed by default on AKS cluster creation
• Others:

[andyzhangx]

it's fixed by https://github.com/kubernetes-sigs/azurefile-csi-driver/pull/1077, currently the fix is only on master branch, we will release v1.22.0 which has this fix.

[nsu700]

Hi, will this feature support cross tenant please? Can we have a sc from another subscription of another tenant?

[andyzhangx]

Hi, will this feature support cross tenant please? Can we have a sc from another subscription of another tenant?

@nsu700 that depends on which cluster identity you are using. if you aks cluster is using managed identity, then it does NOT support cross tenant, if your aks cluster is using service principal, then it supports cross tenant

[andyzhangx]

close this issue since it's already supported in v1.22.0 version with cross subs support.