Deploying to ARO - azure-cloud-provider encoding

[grtn316]

When deploying to ARO, there is already an azure-cloud-provider secret. This secret is plain JSON and not base64'd like it says to do in the docs when creating a new one:

export AZURE_CLOUD_SECRET=cat deploy/cloud.conf | base64 | awk '{printf $0}'; echo

I was updating the file to include:

{
    "cloud":"AzurePublicCloud",
    "tenantId": "$tenantId",
    "subscriptionId": "$subscriptionId",
    "resourceGroup": "$resourceGroupName",
    "location": "$deployRegion",
    "aadClientId": "$clientId",
    "aadClientSecret": "$clientSecret",
    "useManagedIdentityExtension": false,
    "userAssignedIdentityID": "",
    "useInstanceMetadata": true,
    "vmType": "standard",
    "subnetName": "$subnetPrivateEndpointName",
    "vnetName": "$vnetName",
    "vnetResourceGroup": "$resourceGroupName",
    "cloudProviderBackoff": true
}

I initially stored the base64 version which kept failing. After adding the plain JSON object into the secret, the drivers began to work. I did not notice this documented in the ARO docs anywhere so I figured I would mention it.

I am using driver version v1.12.0

[andyzhangx]

do you have the driver controller logs to share, follow https://github.com/kubernetes-sigs/azurefile-csi-driver/blob/master/docs/csi-debug.md#case1-volume-createdelete-issue

[grtn316]

Turns out this is not specific to ARO. I have also discovered it on older version of openshift (4.8.42) and thinking its related to the image versions instead. For example: I just deployed 1.12.0 and am seeing this in the driver controller logs:

I0728 18:22:48.683765       1 azurefile.go:259] driver userAgent: file.csi.azure.com/v1.12.0 gc/go1.17 (amd64-linux) OSS-kubectl
I0728 18:22:48.684216       1 azure.go:71] reading cloud config from secret kube-system/azure-cloud-provider
W0728 18:22:48.696911       1 azure.go:78] InitializeCloudFromSecret: failed to get cloud config from secret kube-system/azure-cloud-provider: failed to parse Azure cloud-config: error unmarshaling JSON: while decoding JSON: json: cannot unmarshal string into Go value of type provider.Config
I0728 18:22:48.696933       1 azure.go:83] could not read cloud config from secret kube-system/azure-cloud-provider
I0728 18:22:48.696951       1 azure.go:86] AZURE_CREDENTIAL_FILE env var set as /etc/kubernetes/cloud.conf
I0728 18:22:48.696978       1 azure.go:101] read cloud config from file: /etc/kubernetes/cloud.conf successfully
I0728 18:22:48.697925       1 azure_auth.go:234] Using AzurePublicCloud environment
I0728 18:22:48.697944       1 azure_auth.go:96] azure: using managed identity extension to retrieve access token
I0728 18:22:48.697948       1 azure_auth.go:107] azure: using System Assigned MSI to retrieve access token

Once I swap it out as plain json, the controller starts using my secret.

[grtn316]

FYI - I have been pinned on 1.12.0 for a while and this started happening recently.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue with /reopen
• Mark this issue as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

[k8s-ci-robot]

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue with /reopen
• Mark this issue as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.