azure-cli

azure-cli copied to clipboard

Describe the bug

Command Name az network application-gateway root-cert create

Errors:

azure.core.pipeline.policies._universal : {
  "status": "Failed",
  "error": {
    "code": "ApplicationGatewayKeyVaultSecretException",
    "message": "Problem occured while accessing and validating KeyVault Secrets associated with Application Gateway '/subscriptions/<removed>/resourceGroups/MC_<removed>-aks-test_westus2/providers/Microsoft.Network/applicationGateways/<removed>appgty-test'. See details below:",
    "details": [
      {
        "code": "ApplicationGatewayTrustedRootCertificateInvalidData",
        "message": "Data for certificate /subscriptions/<removed>/resourceGroups/MC_<removed>-aks-test_westus2/providers/Microsoft.Network/applicationGateways/<removed>-appgty-test/trustedRootCertificates/<removed>backendtls is invalid."
      }
    ]
  }
}

  <removed>File "C:\Users\VSSADM~1\AppData\Local\Temp\pip-unpacked-wheel-0cx4jf6i\azure\core\polling\base_polling.py", line 500, in run
azure.core.exceptions.HttpResponseError: (ApplicationGatewayKeyVaultSecretException) Problem occured while accessing and validating KeyVault Secrets associated with Application Gateway '/subscriptions/<removed>/resourceGroups/MC_<removed>_westus2/providers/Microsoft.Network/applicationGateways/appgty-test'. See details below:

cli.azure.cli.core.azclierror : (ApplicationGatewayKeyVaultSecretException) Problem occured while accessing and validating KeyVault Secrets associated with Application Gateway '/subscriptions/<removed>/resourceGroups/MC_<removed>_westus2/providers/Microsoft.Network/applicationGateways/appgty-test'. See details below:
(ApplicationGatewayKeyVaultSecretException) Problem occured while accessing and validating KeyVault Secrets associated with Application Gateway '/subscriptions/<removed>/resourceGroups/MC_<removed>_westus2/providers/Microsoft.Network/applicationGateways/appgty-test'. See details below:
Event: Cli.PostExecute [<function AzCliLogging.deinit_cmd_metadata_logging at 0x035F3DB0>]
az_command_data_logger : exit code: 1
Command ran in 14.222 seconds (init: 0.230, invoke: 13.993)
telemetry.save : Save telemetry record of length 3321 in cach

To Reproduce:

Steps to reproduce the behavior. Note that argument values have been redacted, as they may contain sensitive information.

• Create KV, add two certs with autorotation on for frontend & backend ssl in RG
• Create AKS cluster with AGIC using this
• Create an identity separately in a different resource group to access KV certs & assign the same as App Gty identity too
• AGIC identity has "Managed Identity Operator" role on this new identity
• create ssl cert with az network application-gateway ssl-cert create -n frontend-tls --gateway-name $APP_GTY_NAME -g $AKS_RG_NAME --key-vault-secret-id $unversionedSecretId
• TRY to create root cert using same az network application-gateway root-cert create --gateway-name $APP_GTY_NAME -g $AKS_RG_NAME --name backend-tls --keyvault-secret $unversionedSecretId2
• The secret ids look like 'https://kvName.vault.azure.net/secrets/backendcert' & 'https://kvName.vault.azure.net/secrets/frontendendcert'

Expected Behavior

Should work fine & pull the cert from KV. Surprisingly the frontend cert works but the backend one fails to be created. I dont want to manually upload the cert as that breaks the auto rotation ideally.

Environment Summary

Windows-10-10.0.19041-SP0
Python 3.6.8
Installer: MSI

azure-cli 2.17.1 *

Extensions:
account 0.2.1
aks-preview 0.4.72
application-insights 0.1.13
kusto 0.2.0

Additional Context

Identity assigned to App Gty: az network application-gateway identity show --gateway-name $APP_GTY_NAME -g $AKS_RG_NAME { "principalId": null, "tenantId": null, "type": "userAssigned", "userAssignedIdentities": { "/subscriptions/{removed}/resourcegroups/test/providers/Microsoft.ManagedIdentity/userAssignedIdentities/appgty-ssl-identity-test": { "clientId": "ebb81e58-{removed}", "principalId": "5d09d269-{removed}" } } } KV Access: appgty-ssl-identity-test has access to get & list for all 3 secret, cert & key. And KV is set for all network access. Also to note the cert in KV is provided by a private CA authority.

Feb 11 '21 22:02 sanjaydebnath

network

Feb 12 '21 00:02 yonzhan

Ok my certificate in AKV for root-cert is a chain certificate. I see that the app gty root cert has to be cer file for the root certificate of my chain cert. Is there a different way of enabling auto rotation for root-cert in that case.... rather than manually uploading? My app gty is WAF_v2.

Feb 12 '21 09:02 sanjaydebnath

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @aznetsuppgithub.

azure.core.pipeline.policies._universal : {
  "status": "Failed",
  "error": {
    "code": "ApplicationGatewayKeyVaultSecretException",
    "message": "Problem occured while accessing and validating KeyVault Secrets associated with Application Gateway '/subscriptions/<removed>/resourceGroups/MC_<removed>-aks-test_westus2/providers/Microsoft.Network/applicationGateways/<removed>appgty-test'. See details below:",
    "details": [
      {
        "code": "ApplicationGatewayTrustedRootCertificateInvalidData",
        "message": "Data for certificate /subscriptions/<removed>/resourceGroups/MC_<removed>-aks-test_westus2/providers/Microsoft.Network/applicationGateways/<removed>-appgty-test/trustedRootCertificates/<removed>backendtls is invalid."
      }
    ]
  }
}

  <removed>File "C:\Users\VSSADM~1\AppData\Local\Temp\pip-unpacked-wheel-0cx4jf6i\azure\core\polling\base_polling.py", line 500, in run
azure.core.exceptions.HttpResponseError: (ApplicationGatewayKeyVaultSecretException) Problem occured while accessing and validating KeyVault Secrets associated with Application Gateway '/subscriptions/<removed>/resourceGroups/MC_<removed>_westus2/providers/Microsoft.Network/applicationGateways/appgty-test'. See details below:

cli.azure.cli.core.azclierror : (ApplicationGatewayKeyVaultSecretException) Problem occured while accessing and validating KeyVault Secrets associated with Application Gateway '/subscriptions/<removed>/resourceGroups/MC_<removed>_westus2/providers/Microsoft.Network/applicationGateways/appgty-test'. See details below:
(ApplicationGatewayKeyVaultSecretException) Problem occured while accessing and validating KeyVault Secrets associated with Application Gateway '/subscriptions/<removed>/resourceGroups/MC_<removed>_westus2/providers/Microsoft.Network/applicationGateways/appgty-test'. See details below:
Event: Cli.PostExecute [<function AzCliLogging.deinit_cmd_metadata_logging at 0x035F3DB0>]
az_command_data_logger : exit code: 1
Command ran in 14.222 seconds (init: 0.230, invoke: 13.993)
telemetry.save : Save telemetry record of length 3321 in cach

Windows-10-10.0.19041-SP0
Python 3.6.8
Installer: MSI

azure-cli 2.17.1 *

Extensions:
account 0.2.1
aks-preview 0.4.72
application-insights 0.1.13
kusto 0.2.0

Mar 18 '21 08:03 ghost

Hi@service team, according to the description, it seems need support from service to allow this kind of certificate.

Mar 18 '21 09:03 msyyc

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @appgwsuppgithub.

azure.core.pipeline.policies._universal : {
  "status": "Failed",
  "error": {
    "code": "ApplicationGatewayKeyVaultSecretException",
    "message": "Problem occured while accessing and validating KeyVault Secrets associated with Application Gateway '/subscriptions/<removed>/resourceGroups/MC_<removed>-aks-test_westus2/providers/Microsoft.Network/applicationGateways/<removed>appgty-test'. See details below:",
    "details": [
      {
        "code": "ApplicationGatewayTrustedRootCertificateInvalidData",
        "message": "Data for certificate /subscriptions/<removed>/resourceGroups/MC_<removed>-aks-test_westus2/providers/Microsoft.Network/applicationGateways/<removed>-appgty-test/trustedRootCertificates/<removed>backendtls is invalid."
      }
    ]
  }
}

  <removed>File "C:\Users\VSSADM~1\AppData\Local\Temp\pip-unpacked-wheel-0cx4jf6i\azure\core\polling\base_polling.py", line 500, in run
azure.core.exceptions.HttpResponseError: (ApplicationGatewayKeyVaultSecretException) Problem occured while accessing and validating KeyVault Secrets associated with Application Gateway '/subscriptions/<removed>/resourceGroups/MC_<removed>_westus2/providers/Microsoft.Network/applicationGateways/appgty-test'. See details below:

cli.azure.cli.core.azclierror : (ApplicationGatewayKeyVaultSecretException) Problem occured while accessing and validating KeyVault Secrets associated with Application Gateway '/subscriptions/<removed>/resourceGroups/MC_<removed>_westus2/providers/Microsoft.Network/applicationGateways/appgty-test'. See details below:
(ApplicationGatewayKeyVaultSecretException) Problem occured while accessing and validating KeyVault Secrets associated with Application Gateway '/subscriptions/<removed>/resourceGroups/MC_<removed>_westus2/providers/Microsoft.Network/applicationGateways/appgty-test'. See details below:
Event: Cli.PostExecute [<function AzCliLogging.deinit_cmd_metadata_logging at 0x035F3DB0>]
az_command_data_logger : exit code: 1
Command ran in 14.222 seconds (init: 0.230, invoke: 13.993)
telemetry.save : Save telemetry record of length 3321 in cach

Windows-10-10.0.19041-SP0
Python 3.6.8
Installer: MSI

azure-cli 2.17.1 *

Extensions:
account 0.2.1
aks-preview 0.4.72
application-insights 0.1.13
kusto 0.2.0

Mar 18 '21 18:03 ghost

Hi, I'm having the same issue. Adding regular ssl-certs from key vault works, but adding a root-cert does not. Maybe it's related to the fact that the root-cert cannot be added as "certificate" to key vault (because there is no key), but instead must be added as "secret"? In any case, the only way I found for me was to add the root cert to the gateway by direct upload instead of a key vault reference, which I consider an intermediate "workaround", but not a soluion.

Aug 11 '21 10:08 Phrow

same problem - the az cli has the --keyvault secret argument for az network application-gateway root-cert create command but I get Code: ApplicationGatewayTrustedRootCertificateInvalidData

If I use the file directly instead of the secret in keyvault it works

And I have to get it from a keyvault secret cause apparently you can't add trusted root certificates in Keyvaul certificates

So not sure how the format is once you have it the keyvault secret but I imagine something is not ok there

Nov 04 '21 23:11 adin3d

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @aznetsuppgithub.

azure.core.pipeline.policies._universal : {
  "status": "Failed",
  "error": {
    "code": "ApplicationGatewayKeyVaultSecretException",
    "message": "Problem occured while accessing and validating KeyVault Secrets associated with Application Gateway '/subscriptions/<removed>/resourceGroups/MC_<removed>-aks-test_westus2/providers/Microsoft.Network/applicationGateways/<removed>appgty-test'. See details below:",
    "details": [
      {
        "code": "ApplicationGatewayTrustedRootCertificateInvalidData",
        "message": "Data for certificate /subscriptions/<removed>/resourceGroups/MC_<removed>-aks-test_westus2/providers/Microsoft.Network/applicationGateways/<removed>-appgty-test/trustedRootCertificates/<removed>backendtls is invalid."
      }
    ]
  }
}

  <removed>File "C:\Users\VSSADM~1\AppData\Local\Temp\pip-unpacked-wheel-0cx4jf6i\azure\core\polling\base_polling.py", line 500, in run
azure.core.exceptions.HttpResponseError: (ApplicationGatewayKeyVaultSecretException) Problem occured while accessing and validating KeyVault Secrets associated with Application Gateway '/subscriptions/<removed>/resourceGroups/MC_<removed>_westus2/providers/Microsoft.Network/applicationGateways/appgty-test'. See details below:

cli.azure.cli.core.azclierror : (ApplicationGatewayKeyVaultSecretException) Problem occured while accessing and validating KeyVault Secrets associated with Application Gateway '/subscriptions/<removed>/resourceGroups/MC_<removed>_westus2/providers/Microsoft.Network/applicationGateways/appgty-test'. See details below:
(ApplicationGatewayKeyVaultSecretException) Problem occured while accessing and validating KeyVault Secrets associated with Application Gateway '/subscriptions/<removed>/resourceGroups/MC_<removed>_westus2/providers/Microsoft.Network/applicationGateways/appgty-test'. See details below:
Event: Cli.PostExecute [<function AzCliLogging.deinit_cmd_metadata_logging at 0x035F3DB0>]
az_command_data_logger : exit code: 1
Command ran in 14.222 seconds (init: 0.230, invoke: 13.993)
telemetry.save : Save telemetry record of length 3321 in cach

Windows-10-10.0.19041-SP0
Python 3.6.8
Installer: MSI

azure-cli 2.17.1 *

Extensions:
account 0.2.1
aks-preview 0.4.72
application-insights 0.1.13
kusto 0.2.0

Nov 05 '21 00:11 ghost

I can confirm I'm having this issue too.

I'm able to upload .cer root certificates via clickOps on my App Gateway but cannot automate my deployment from key vault, resulting in:

{ "code": "ApplicationGatewayTrustedRootCertificateInvalidData", "message": "Data for certificate /subscriptions/<subscription_id>/resourceGroups/<resource_group>/providers/Microsoft.Network/applicationGateways/<app_gateway_name>/trustedRootCertificates/<certname> is invalid." }

Nov 10 '21 09:11 pmontanaro

Facing the above issue when the root certificate is referenced from the keyvault

{
"code": "ApplicationGatewayTrustedRootCertificateInvalidData",
"message": "Data for certificate <full-resource-id>trustedRootCertificates/root-cert is invalid."
}

Jul 21 '22 19:07 rahulprabhu22

Hi All,

Today I successfully added a root-cert to an Application Gateway (WAF V2) from a secret stored in a Key Vault.

Key Vault

Add the secret to the key vault. az keyvault secret set --name Internal-CA-Chain-base64 --vault-name My-KeyVault --subscription My-Sub --file Internal-CA-Chain.cer --encoding base64

Internal-CA-Chain.cer is a PEM file with Root and two Intermediates like so:

-----BEGIN CERTIFICATE-----
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
-----END CERTIFICATE-----

In order like so: Issuer: RootCA Subject: RootCA Issuer: RootCA Subject: InterCA Issuer: InterCA Subject: IssuingCA

Once the secret is created retrieve the Secret ID.

$ az keyvault secret list --vault-name My-KeyVault --subscription My-Sub --output json --query "[].{Name:name,ID:id}"
[
  {
    "ID": "https://my-keyvault.vault.azure.net/secrets/Internal-CA-Chain-base64",
    "Name": "Internal-CA-Chain-base64"
  }
]

Application Gateway

Add the root-cert to the application gateway. az network application-gateway root-cert create --keyvault-secret 'https://my-keyvault.vault.azure.net/secrets/Internal-CA-Chain-base64' --name Internal-CA-Chain --gateway-name My-AppGW --resource-group My-ResGroup --subscription My-Sub

List the root-certs installed to the application gateway.

az network application-gateway root-cert list --gateway-name My-AppGW --resource-group My-ResGroup --subscription My-Sub --output json --query "[].{Name:name,ProvisioningState:provisioningState,KeyVaultSecretId:keyVaultSecretId}"
[
  {
    "KeyVaultSecretId": "https://my-keyvault.vault.azure.net/secrets/Internal-CA-Chain-base64",
    "Name": "Internal-CA-Chain",
    "ProvisioningState": "Succeeded"
  }
]

Aug 11 '22 06:08 larwood

I'm still facing this despite trying the above steps. Any updates for this issue?

Nov 10 '23 00:11 toddmacintyre