bicep icon indicating copy to clipboard operation
bicep copied to clipboard
verified publisher icon Azure

Bicepparam error when passing to a string parameter

Open jbtcode opened this issue 1 year ago • 1 comments

Bicep version Bicep CLI version 0.24.24 (5646341b0c)

Describe the bug I'm trying to pass a GUID string generated by the guid() function into a string parameter, but this result in the error:

Error: Code=InvalidTemplate; Message=Deployment template validation failed: 'The provided value for the template parameter 'administratorPassword' is not valid. Expected a value of type 'String, Uri', but received a value of type 'Object'.

When I first compile the .bicepparam file to the .json (ARM template) format, it does work, because the build-param command has already evaluated the guid() function call and stores the string value:

{
  "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#",
  "contentVersion": "1.0.0.0",
  "parameters": {
    "administratorPassword": {
      "value": "ac1f0658-f2ed-5c6a-bce1-56138c8b5f4e"
    }
  }
}

However, the guid() function is documented to return a string so this should work

To Reproduce

Bicep template:

   @description('The administrator password (required for server creation). ')
   @secure()
   @minLength(1)
   param administratorPassword string

Bicepparam file:

   using './template.bicep'
   param administratorPassword = guid('dummy')

Deployment using powershell:

   $paramPath = Join-Path $PSScriptRoot "test.bicepparam"
   $deployParams = @{
        Name = 'test-deployment-dummy'
        ResourceGroupName = '<resourcegroup here>'
        TemplateFile = Join-Path $PSScriptRoot "template.bicep"
        TemplateParameterFile = $paramPath
        InformationAction = 'Continue'
        ErrorAction = "Stop"         
        ErrorVariable = "deployErrors" 
    }
    $deployOutput = New-AzResourceGroupDeployment @deployParams  
    Write-Output "Completed with ProvisioningState: $($deployOutput.ProvisioningState)

Output:

New-AzResourceGroupDeployment: D:\bicep\param-issue\deploy.ps1:29:21
Line |
  29 |      $deployOutput = New-AzResourceGroupDeployment @deployParams
     |                      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     | 11:05:31 - Error: Code=InvalidTemplate; Message=Deployment template validation failed: 'The provided value for the template parameter 'administratorPassword' is not valid. Expected  
     | a value of type 'String, Uri', but received a value of type 'Object'. Please see https://aka.ms/arm-create-parameter-file for usage details.'.

However it works when first compiling the bicepparam file to json:

   $paramPath = Join-Path $PSScriptRoot "test.bicepparam"

   # workaround: compile bicepparam to json
   $paramPathJson = Join-Path $PSScriptRoot "test.json"
   . bicep build-params $paramPath --outfile $paramPathJson

   $deployParams = @{
        Name = 'test-deployment-dummy'
        ResourceGroupName = '<resourcegroup here>'
        TemplateFile = Join-Path $PSScriptRoot "template.bicep"
        TemplateParameterFile = $paramPath
        InformationAction = 'Continue'
        ErrorAction = "Stop"         
        ErrorVariable = "deployErrors" 
    }
    $deployOutput = New-AzResourceGroupDeployment @deployParams  
    Write-Output "Completed with ProvisioningState: $($deployOutput.ProvisioningState)

Output:

Completed with ProvisioningState: Succeeded

jbtcode avatar Feb 01 '24 10:02 jbtcode

Potentially related to #12481.

jeskew avatar Feb 01 '24 15:02 jeskew

This also seems to apply when the template file uses @secure() and you try to pass the value as "override" in the CLI. Could not get it to work in any combination I tried using Bicep CLI version 0.26.54 (5e20b29b58)

test.bicep

@secure()
param password string

test.bicepparam

using 'test.bicep'

@secure() // doesn't matter
param password = 'hello'

When trying to deploy using an override for password you'll get an error

Without @secure in .bicepparam:

  1. New-AzSubscriptionDeployment -TemplateParameterFile .\test.bicepparam -password "hi"

New-AzDeployment: Cannot bind parameter 'password'. Cannot convert the value of type "System.String" to type "System.Security.SecureString".

  1. New-AzSubscriptionDeployment -TemplateParameterFile .\test.bicepparam -password new NetworkCredential("", "hi").SecurePassword

Error BCP033: Expected a value of type "string" but the provided value is of type "object".

With @secure() in .bicepparam:

  1. New-AzSubscriptionDeployment -TemplateParameterFile .\test.bicepparam -password "hi"

New-AzDeployment: Cannot bind parameter 'password'. Cannot convert the value of type "System.String" to type "System.Security.SecureString".

  1. New-AzSubscriptionDeployment -TemplateParameterFile .\test.bicepparam -password new NetworkCredential("", "hi").SecurePassword

Error BCP033: Expected a value of type "string" but the provided value is of type "object".

Dropping @secure everywhere removes the error

quality-leftovers avatar Mar 25 '24 08:03 quality-leftovers

When consuming a bicepparam file I experience the same behavior @quality-leftovers and @jbtcode have described.

If I compile the bicepparam file to a json parameter file first, I can provide a SecureString as a value to an inline parameter and get the expected behavior.

eamreyes avatar May 30 '24 23:05 eamreyes

We have been running deployments in Azure DevOps with a bicepparam supplying an empty string to a bicep template and then overriding it in the New-AzSubscriptionDeployment with a securestring like this for months:

main.bicepparam:

using './main.bicep'
param clientSecret = ''

main.bicep:

@description('Secret for the WebAPI application')
@secure()
param clientSecret string

Deployment

$params = @{
          clientSecret = ConvertTo-SecureString -String "${{parameters.clientSecret}}" -AsPlainText -Force
          deploymentName = "${{parameters.deploymentName}}"
          location = "${{parameters.location}}"
          TemplateFile = "main.bicep"
          TemplateParameterFile = "main.bicepparam"
          }
New-AzSubscriptionDeployment @params

Suddenly, this stopped working yesterday. It also stopped working when running locally.

 Error BCP033: Expected a value of type "string" but the provided value is of type "object"

Removing the @secure() descriptor from main.bicep and providing a clear text password makes the deployment work again.

Bicep version before and after the issue occurred: Bicep CLI version 0.28.1

Az module versions testet: 10.3.0 12.0.0

For now running without the secure decorator makes it possible to continue deployments, but we would prefer to use the secure decorator.

I would be glad to provide additional information, if that is of any help in solving this issue.

Agazoth avatar Jul 09 '24 09:07 Agazoth

Hmm, I much preferred this: image to what everyone can probably imagine is displayed now ...

@alex-frankel do you have any idea why this stopped working?

Agazoth avatar Jul 09 '24 09:07 Agazoth

@Agazoth just wanted to clarify - did the issue appear when upgrading the Bicep CLI, or the Az modules?

Please could you confirm:

  • Version(s) of Bicep & Az used where you found this was working
  • Version(s) of Bicep & Az used where you found this wasn't working

For this comment, would you mind sharing the details of the workaround you had to make to your PowerShell to "provide the clear text password"?

Removing the @secure() descriptor from main.bicep and providing a clear text password makes the deployment work again.

I also want to make sure you're aware that removing @secure() for a sensitive parameter is a potential security risk - it means anyone with reader access in your subscription will be able to obtain the parameter from deployment history.

anthony-c-martin avatar Jul 09 '24 14:07 anthony-c-martin

Hi @anthony-c-martin

Working untill yesterday:

Bicep CLI version 0.28.1 Az module version 10.3.0

Same versions not working after 13.00 yesterday.

I have the pipeline history to prove it, but I cannot post it here.

Agazoth avatar Jul 09 '24 14:07 Agazoth

@Agazoth So just to clarify - from what you can see, this wasn't related to a change in Bicep CLI or Az modules, rather some other (unknown) change on the build agent?

anthony-c-martin avatar Jul 09 '24 15:07 anthony-c-martin

Yes, that is correct. For months it worked on

Current agent version: '3.241.0'
Operating System
Ubuntu
22.04.4
LTS
Runner Image
Runner Image Provisioner
Current image version: '20240630.1.0'

for months. Then yesterday - on the same runner with the same versions of bicep and Az module, the error occours.

Agazoth avatar Jul 09 '24 15:07 Agazoth

That is puzzling! Thank you for confirming.

anthony-c-martin avatar Jul 09 '24 15:07 anthony-c-martin

@anthony-c-martin I found some time to clarify:

At 2024-07-08T06:10:12.2196359Z we started a pipeline run that concluded successfully. It had the following configuration:

2024-07-08T06:20:54.9088280Z Agent name: 'Hosted Agent'
2024-07-08T06:20:54.9088551Z Agent machine name: 'fv-az457-632'
2024-07-08T06:20:54.9088678Z Current agent version: '3.241.0'
2024-07-08T06:20:54.9123548Z ##[group]Operating System
2024-07-08T06:20:54.9123706Z Ubuntu
2024-07-08T06:20:54.9123781Z 22.04.4
2024-07-08T06:20:54.9123866Z LTS
2024-07-08T06:20:54.9123949Z ##[endgroup]
2024-07-08T06:20:54.9124040Z ##[group]Runner Image
2024-07-08T06:20:54.9124156Z Image: ubuntu-22.04
2024-07-08T06:20:54.9124255Z Version: 20240630.1.0
2024-07-08T06:20:54.9124467Z Included Software: https://github.com/actions/runner-images/blob/ubuntu22/20240630.1/images/ubuntu/Ubuntu2204-Readme.md
2024-07-08T06:20:54.9124723Z Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu22%2F20240630.1
2024-07-08T06:20:54.9124870Z ##[endgroup]
2024-07-08T06:20:54.9124981Z ##[group]Runner Image Provisioner
2024-07-08T06:20:54.9125224Z 2.0.370.1
2024-07-08T06:20:54.9125326Z ##[endgroup]
2024-07-08T06:20:54.9126123Z Current image version: '20240630.1.0'
2024-07-08T06:20:54.9127515Z Agent running as: 'vsts'
2024-07-08T06:20:54.9163667Z Prepare build directory.
2024-07-08T06:20:54.9438758Z Set build variables.
2024-07-08T06:20:54.9475984Z Download all required tasks.
2024-07-08T06:20:54.9645996Z Downloading task: PowerShell (2.239.1)
2024-07-08T06:20:55.7517025Z Downloading task: AzurePowerShell (5.241.4)
2024-07-08T06:20:57.0782178Z Downloading task: AzureCLI (2.241.2)
2024-07-08T06:21:02.1859336Z Bicep CLI version 0.28.1 (ba1e9f8c1e)
2024-07-08T06:21:03.0718082Z Task         : Azure PowerShell
2024-07-08T06:21:03.0718147Z Description  : Run a PowerShell script within an Azure environment
2024-07-08T06:21:03.0718235Z Version      : 5.241.4
2024-07-08T06:21:04.8306543Z Az version 10.3.0 not avaiable locally on the agent. Downloading dynamically.

No error her. The template gets deployed with a nice log:

2024-07-08T06:26:00.9224127Z                           clientSecret                SecureString             
2024-07-08T06:26:00.9225089Z                             null

At 2024-07-08T12:30:20.9719966Z the pipeline stopped working with this configuration:

2024-07-08T12:30:21.1425434Z Agent name: 'Hosted Agent'
2024-07-08T12:30:21.1425737Z Agent machine name: 'fv-az355-737'
2024-07-08T12:30:21.1425884Z Current agent version: '3.241.0'
2024-07-08T12:30:21.1457930Z ##[group]Operating System
2024-07-08T12:30:21.1458089Z Ubuntu
2024-07-08T12:30:21.1458172Z 22.04.4
2024-07-08T12:30:21.1458267Z LTS
2024-07-08T12:30:21.1458354Z ##[endgroup]
2024-07-08T12:30:21.1458452Z ##[group]Runner Image
2024-07-08T12:30:21.1458584Z Image: ubuntu-22.04
2024-07-08T12:30:21.1458693Z Version: 20240630.1.0
2024-07-08T12:30:21.1458929Z Included Software: https://github.com/actions/runner-images/blob/ubuntu22/20240630.1/images/ubuntu/Ubuntu2204-Readme.md
2024-07-08T12:30:21.1459239Z Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu22%2F20240630.1
2024-07-08T12:30:21.1459406Z ##[endgroup]
2024-07-08T12:30:21.1459514Z ##[group]Runner Image Provisioner
2024-07-08T12:30:21.1459775Z 2.0.370.1
2024-07-08T12:30:21.1459880Z ##[endgroup]
2024-07-08T12:30:21.1460659Z Current image version: '20240630.1.0'
2024-07-08T12:30:21.1462124Z Agent running as: 'vsts'
2024-07-08T12:30:21.1499577Z Prepare build directory.
2024-07-08T12:30:21.1767244Z Set build variables.
2024-07-08T12:30:21.1803869Z Download all required tasks.
2024-07-08T12:30:21.1972078Z Downloading task: PowerShell (2.239.1)
2024-07-08T12:30:22.0576380Z Downloading task: AzurePowerShell (5.241.4)
2024-07-08T12:30:23.6468922Z Downloading task: AzureCLI (2.241.2)
2024-07-08T12:30:30.1053770Z Bicep CLI version 0.28.1 (ba1e9f8c1e)
2024-07-08T12:30:31.0518911Z Task         : Azure PowerShell
2024-07-08T12:30:31.0518984Z Description  : Run a PowerShell script within an Azure environment
2024-07-08T12:30:31.0519077Z Version      : 5.241.4
2024-07-08T12:30:33.0081118Z Az version 10.3.0 not avaiable locally on the agent. Downloading dynamically.

The error generated:

Line |
  15 |  New-AzSubscriptionDeployment @params
     |  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     | Cannot retrieve the dynamic parameters for the cmdlet. Object reference
     | not set to an instance of an object.

Updating to Az module 12.0.0 generates a different error as can be seen in this run

2024-07-09T08:25:12.0066563Z Agent machine name: 'fv-az368-975'
2024-07-09T08:25:12.0066765Z Current agent version: '3.241.0'
2024-07-09T08:25:12.0109183Z ##[group]Operating System
2024-07-09T08:25:12.0109417Z Ubuntu
2024-07-09T08:25:12.0109534Z 22.04.4
2024-07-09T08:25:12.0109633Z LTS
2024-07-09T08:25:12.0109762Z ##[endgroup]
2024-07-09T08:25:12.0109897Z ##[group]Runner Image
2024-07-09T08:25:12.0110045Z Image: ubuntu-22.04
2024-07-09T08:25:12.0110204Z Version: 20240630.1.0
2024-07-09T08:25:12.0110511Z Included Software: https://github.com/actions/runner-images/blob/ubuntu22/20240630.1/images/ubuntu/Ubuntu2204-Readme.md
2024-07-09T08:25:12.0110905Z Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu22%2F20240630.1
2024-07-09T08:25:12.0111164Z ##[endgroup]
2024-07-09T08:25:12.0111307Z ##[group]Runner Image Provisioner
2024-07-09T08:25:12.0111628Z 2.0.370.1
2024-07-09T08:25:12.0111766Z ##[endgroup]
2024-07-09T08:25:12.0112727Z Current image version: '20240630.1.0'
2024-07-09T08:25:12.0114229Z Agent running as: 'vsts'
2024-07-09T08:25:12.0160110Z Prepare build directory.
2024-07-09T08:25:12.0517933Z Set build variables.
2024-07-09T08:25:12.0560963Z Download all required tasks.
2024-07-09T08:25:12.0781184Z Downloading task: PowerShell (2.239.1)
2024-07-09T08:25:13.0740191Z Downloading task: AzurePowerShell (5.242.0) <-- UPDATED!
2024-07-09T08:25:16.3617840Z Downloading task: AzureCLI (2.242.0) <-- UPDATED!
2024-07-09T08:25:24.2561060Z Bicep CLI version 0.28.1 (ba1e9f8c1e)
2024-07-09T08:25:25.3757996Z Task         : Azure PowerShell
2024-07-09T08:25:25.3758080Z Description  : Run a PowerShell script within an Azure environment
2024-07-09T08:25:25.3758195Z Version      : 5.242.0 <-- UPDATED!
2024-07-09T08:25:28.0234068Z Az version 12.0.0 not avaiable locally on the agent. Downloading dynamically.

And we then get this error

Line |
  15 |  New-AzSubscriptionDeployment @params
     |  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     | /home/vsts/work/1/s/infra/Deploy/dev.bicepparam(13,22) : Error BCP033:
     | Expected a value of type "string" but the provided value is of type
     | "object".

In short, we go from working to Cannot retrieve the dynamic parameters for the cmdlet on the same AzurePowerShell/Az version.

Upgrading to a newer AzurePowerShell/Az version introduces the more intelligble error Expected a value of type "string" but the provided value is of type "object".

We did a run on the new Azure PowerShell 5.242.0 vewrsion. That also produces the old error: Cannot retrieve the dynamic parameters for the cmdlet

Running the updated Azure PowerShell with the new Az module without the @secure() attribute lets us deploy the template, but it also logs the secret in the pipeline run. This WILL be found by the auditors of our customers.

I am very interested in finding a solution for this issue and I am available for any kind of testing and further investigation. I cannot share any more information from our logs in public, but I would be happy to proceed under less public conditions.

Agazoth avatar Jul 10 '24 05:07 Agazoth

So I think the issue is that when the SecureString parameter is passed from PowerShell to Bicep CLI, it is being serialized naively, rather than converting the value back to a string.

With some debug logging enabled, I see the following being passed:

{"clientSecret":{"Length":5}}

I think this explains the error Expected a value of type "string" but the provided value is of type "object". that you see. To address this, we'd need to make a fix here to convert back to string before serializing.

What I can't explain is why this would have suddenly started happening - as you've demonstrated, it doesn't appear to be related to a change in AzPwsh or Bicep CLi...

I would recommend as a more secure workaround to keep the @secure() in Bicep, and simply use clientSecret = "${{parameters.clientSecret}}" instead of clientSecret = ConvertTo-SecureString -String "${{parameters.clientSecret}}" -AsPlainText -Force when passing in the parameter. There's no way to avoid the string value being sent on the wire, but the @secure() means the deployment service will understand that it's a secure value and handle appropriately.

anthony-c-martin avatar Jul 18 '24 10:07 anthony-c-martin

Great news about the fix @anthony-c-martin! I'm looking forward to it.

I tried adding @secure() back into the main.bicep template and keeping clientSecret = "${{parameters.clientSecret}}"but gives me this error when deploying:

New-AzDeployment: Cannot bind parameter 'clientSecret'. Cannot convert the value of type "System.String" to type "System.Security.SecureString".

Agazoth avatar Jul 18 '24 11:07 Agazoth

Bicep CLI version 0.29.47 seems to include the fix. Can you confirm @anthony-c-martin ?

Agazoth avatar Aug 14 '24 10:08 Agazoth

@Agazoth the fix was made to Azure PowerShell (not Bicep), and part of the v12.2.0 release.

It should work with Bicep CLI 0.29.47, but should also work with earlier releases.

anthony-c-martin avatar Aug 14 '24 15:08 anthony-c-martin