bicep
bicep copied to clipboard
Azure
Allow deployment scripts to use private virtual networking
Bicep version 0.4.1318
Describe the bug
- first we have created a user assigned Identity. this identity has access on key vault secrets. 2.Then we have created private end point for this key Vault.
- we have chosen selected network inside firewall and virtual networks for key Vault settings.
- finally when we are running deployment script to fetch the secrets inside key vault. it is giving errors.
below I have pasted deployment script and error message.
**Deployment Script Code: **
param location string
param name string
param utcValue string = utcNow()
param MEUAIdentityID string
param accessKeyVaultName string
param accessSecretName string
resource genrateRandomPassword 'Microsoft.Resources/deploymentScripts@2020-10-01' = {
name: name
location: location
identity: {
type: 'UserAssigned'
userAssignedIdentities: {
'${MEUAIdentityID}': {}
}
}
kind: 'AzurePowerShell'
properties: {
forceUpdateTag: utcValue
azPowerShellVersion: '6.6'
timeout: 'PT30M'
arguments: ' -accessKeyVaultName ${accessKeyVaultName} -accessSecretName ${accessSecretName}'
scriptContent: '''
param(
[string] [Parameter(Mandatory=$true)] $accessKeyVaultName,
[string] [Parameter(Mandatory=$true)] $accessSecretName
)
$ErrorActionPreference = 'Stop'
$DeploymentScriptOutputs = @{}
$uppercase = "ABCDEFGHKLMNOPRSTUVWXYZ".tochararray()
$lowercase = "abcdefghiklmnoprstuvwxyz".tochararray()
$number = "0123456789".tochararray()
$special = "$%/()=?}{@#*+!".tochararray()
$password =($uppercase | Get-Random -count 1) -join ""
$password +=($lowercase | Get-Random -count 5) -join ""
$password +=($number | Get-Random -count 1) -join ""
$password +=($special | Get-Random -count 1) -join ""
$passwordarray=$password.tochararray()
$scrambledpassword=($passwordarray | Get-Random -Count 8) -join ""
$secretvalue = ConvertTo-SecureString $scrambledpassword -AsPlainText -Force
$accessSecret = Get-AzKeyVaultSecret -VaultName $accessKeyVaultName -Name $accessSecretName -AsPlainText
if([string]::IsNullOrEmpty($accessSecret)){
Set-AzKeyVaultSecret -VaultName $accessKeyVaultName -Name $accessSecretName -SecretValue $secretvalue
}
'''
cleanupPreference: 'OnSuccess'
retentionInterval: 'P1D'
}
}
To Reproduce
-
we have to create private end point for keyVault inside virtual network.
-
Then we have to deploy deployment script
Additional context error Message "message": "{\r\n "status": "failed",\r\n "error": {\r\n "code": "ResourceDeploymentFailure",\r\n "message": "The resource operation completed with terminal provisioning state 'failed'.",\r\n "details": [\r\n {\r\n "code": "DeploymentScriptError",\r\n "message": "The provided script failed with the following error:\r\nMicrosoft.Azure.KeyVault.Models.KeyVaultErrorException: Operation returned an invalid status code 'Forbidden'\nCode: Forbidden\nMessage: Client address is not authorized and caller is not a trusted service.\r\nClient address: 20.84.24.106
Today, it is not possible to create a deployment script in a private network. We don't have an ETA for when we will be able to enable this.
Same problem here, I'm using biceps with deploymentScripts resource for accessing a key vault whose network access is restricted...
The thing is that DeploymentScripts are run inside a random container managed completely by Azure, therefore end-users have no control over it. The public IP used by the random container change between different runs. Thus, there's no way of adding the IP to the key vault's firewall...
Disabling the Firewall and allowing all public internet traffic to go through is not a possibility for a production environment, as it decreases security...
Key vault configuration:
Deployment Script:
resource addCertificateToKeyVault 'Microsoft.Resources/deploymentScripts@2020-10-01' = {
name: 'dscript-addcertificatetokeyvault'
location: location
kind: 'AzurePowerShell'
properties: {
azPowerShellVersion: azPowerShellVersion
retentionInterval: 'P0DT26H'
timeout: 'P0DT5H'
environmentVariables: [
{
name: 'TenantId'
secureValue: tenant().tenantId
}
{
name: 'SubscriptionId'
secureValue: subscription().subscriptionId
}
{
name: 'ServicePrincipalId'
secureValue: ServicePrincipalAppId
}
{
name: 'ServicePrincipalPassword'
secureValue: ServicePrincipalPassword
}
]
arguments: '-KeyVaultName ${keyVaultName} -CertificateName ${SSLCertificateName} -Certificate ${SSLCertificate} -Base64'
scriptContent: loadTextContent('powershell/Add-CertificateToKeyVault.ps1')
Error displayed:
The deployment 'XXXXX' failed with error(s).
| Showing 3 out of 3 error(s). Status Message: The provided
| script failed with the following error:
| Microsoft.Azure.KeyVault.Models.KeyVaultErrorException:
| Operation returned an invalid status code 'Forbidden' Code:
| Forbidden Message: Client address is not authorized and caller
| is not a trusted service. Client address: xx.xx.xx.xxx
| Caller: xxxxx
The service principal is allowed in the key vault access policies:
I added my feedback because the first comment was in April and we're in September, that way I can add more information to the issue.
Thank you!
As a work around we wrapped our PS script in this section
Write-Host "Adding the current public ip to the key vault allow list"
$publicIp = "$(Invoke-WebRequest https://api.ipify.org)/32"
Add-AzKeyVaultNetworkRule -VaultName $keyVaultName -IpAddressRange $publicIp
# Do what you want with secrets, certs
Write-Host "Removing current public ip address from allow list"
Remove-AzKeyVaultNetworkRule -VaultName $keyVaultName -IpAddressRange $publicIp
We expect to ship a new API version of deployment scripts that will enable this in less than two months. We are in the late stages of implementation.
@alex-frankel glad to hear this is going to be addressed! Is there a more recent update about when this will be available?
We are waiting for the backend deployment to complete before we can try testing the feature end to end. If everything goes well, we expect it to be live by end of November.
cc @subha-sam
Unfortunately, we had a last-minute change to our holiday deployment freeze which is delaying the rollout and validation of this feature. We are now targeting end of December.
Hi @majastrz - Could you explain how this will work. Our use case is essentially we have a Key Vault with the Allow public access from specific virtual networks and IP addresses. Does that mean with this update, we will be able to create a deploymentScript that deploys a container instance inside of a Virtual network which would then be allowed to access our KeyVault if that virtual network is within the 'Allow' rules?
Thanks,
Yes, that should be supported.
Deployment Scripts use Azure Container Instances (ACI) to run the provided scripts in a container. This change is adding a new API version to Deployment Scripts that exposes a new property called subnetIds that will be passed to the ACI resource created by Deployment Scripts. Here are also the ACI docs for reference: https://learn.microsoft.com/en-us/azure/container-instances/container-instances-vnet
@majastrz is there any update regarding the release date of this feature?
The last deployment needed to enable this will start on 1/9. If everything proceeds properly, then it will be fully rolled out by 1/16.
Hey folks -- unfortunately, after progressing through some of the deployment we identified an issue with the implementation for this feature that we did not account for. This means that the feature is on hold until we complete the investigation into possible fixes.
I know this is not great news, but as always, we want to be as transparent with you as possible. I am hoping we can share an update sometime next week.
Hey, we have the same challenge on this one. Any update on the issue and potential deployment? @alex-frankel
Hey All,
Apologies about the delay here. We had to revisit the design to comply with a set of networking permissions that we hadn't accounted for. This has resulted in changes on the requirements to use private networking with deployment scripts.
Please take a look and let us know if any of them are blockers for using this capability
New Requirements:
- Must use a user-assigned managed identity
- The identity must have been granted the Storage File Data SMB Share Contributor and Storage Account Contributor roles
- Must use an existing storage account. At least initially, we won't be able to support Deployment Scripts creating the storage account on your behalf.
- Storage account must have granted access to "Allow all services" enabled
-
- Cannot use a storage account key to communicate with the storage account. We will need to use Azure RBAC and will block any requests that try to use private networking AND storage account keys.
In terms of ETA, I don't want to overpromise. The work is underway, but will have to follow as get deeper into the updated implementation.
I don't believe these are blockers for me. The allow all services one isn't ideal, but we can live with it.
I believe this would be fine in our setup as well. 😀 Thanks for the update.
@alex-frankel Any update on this. Patiently awaiting this feature! I see it's listed in milestone v0.17 due in May. Will we have to wait that long?
@craigeaw - most likely at least that long. The work is in progress and will share a more precise ETA as the work gets closer to being completed.
Any update on ETA, we have the same issue. Would like to bind the ACI to a subnet to perform some tasks on private resource (Keyvault and Sysnapse).
No meaningful updates other than we are still working on it! There are a lot of external dependencies/approvals/deployments we need to wait on as we make progress on this one, so that is why it has been slow. Hoping we can get this out in June/July, but we will see. As always, apologies for the delay on this one and thanks for everybody's patience.
I'm not going to lie and say I'm patiently waiting for this, but I'm excited about the possibilities that this feature will open up.
