bicep icon indicating copy to clipboard operation
bicep copied to clipboard

loadTextContent() - support variable or parameter as input

Open slavizh opened this issue 3 years ago • 52 comments

Is your feature request related to a problem? Please describe.

I would like for loadTextContent() to support variable or parameter as input parameter. What I would like to offer is for the end user to specify text from file or to use the default file available. Currently if I use the below syntax I get error: The value must be a compile-time constant.bicep(BCP032)

I know that function is being used at compile time so this will probably require some other approach and even changes in template spec for supporting it there as well.

var alertRules = {
  memoryLow: {
    description: empty(bastionHostMonitoring.alertRules.memoryLow.descriptionFilePath) ? loadTextContent('alert-descriptions/memory-low.txt') : loadTextContent(bastionHostMonitoring.alertRules.memoryLow.descriptionFilePath)
  }
}

slavizh avatar Jul 29 '21 09:07 slavizh

We cannot do this with a param since we won't know the value at compile time. In other words, if you were to run bicep build with a param argument, we'd have no idea which file to "load".

In theory, this should be possible with variables, but my understanding is it requires more of #444 implemented

alex-frankel avatar Jul 29 '21 15:07 alex-frankel

Very interested in this thread.

The ability to load another file inside a var would open a load of options around not hardcoded 10s or 100s are variables to reference existing file to load data

var manifest = json(loadTextContent('./data/manifest.json'))

var itemsArray = [ for item in manifest : {
  data: json(loadTextContent('./data/${item.dataFile}'))
  parameters: json(loadTextContent('./data/${item.parametersFile}'))
}

In theory, this should all be available at compile time as all files are static and nothing is dynamically generated. The manifest object is be generated. So, the itemsArray should be compiled off manifest as well at compile time. Depending on how the complier interrupts order of operation, right?

mblant avatar Dec 16 '21 22:12 mblant

#3607 seems related to this @mblant, but it is not the same ask

alex-frankel avatar Dec 22 '21 18:12 alex-frankel

I removed the string interpellation and just put the file path in the manifest file and it worked. So, I then assumed it is the interpellation causing the compile time error.

But then, after three or four runs in my test environment, it’s back to the compile time error “The value must be a compile-time constant. bicep(BCP032)” Not sure what is up in the compiler that it let it run several times (and it actually worked!!) only to go back to throwing the error with no changes? Happy to get some more data, if someone can give a pointer on where to start.

az bicep version Bicep CLI version 0.4.1124 (66c84c8ee5)

mblant avatar Jan 05 '22 23:01 mblant

Interesting. No errors if there is only a single element in the manifest.json. Add additional elements and back to failure.

mblant avatar Jan 06 '22 01:01 mblant

Hi, I have the same error when providing more than one file location getting the "The value must be a compile-time constant." error.Please let us know how to fix this issue

var stringArray = [ './policies/policy_1.json' ]

var policies = [for i in range(0, length(stringArray)): json(loadTextContent(stringArray[i]))]

leechan-bicep avatar Jan 17 '22 12:01 leechan-bicep

Repro for triage:

works:

var stringArray = [
  './policies/policy_1.json'
]
  
var policies = [for i in range(0, length(stringArray)): json(loadTextContent(stringArray[i]))]

doesn't work:

var stringArray = [
  './policies/policy_1.json'
  './policies/policy_2.json'
]
  
var policies = [for i in range(0, length(stringArray)): json(loadTextContent(stringArray[i]))]

alex-frankel avatar Jan 24 '22 03:01 alex-frankel

The reason this works is because the type of a single-item string array gets simplified down to a string literal, which the function can handle. It's not something we intentionally designed and in fact we should probably block this scenario. We are going to take a look at validating this better.

alex-frankel avatar Jan 26 '22 20:01 alex-frankel

@miqm will break out this last comment and track the fix with a separate issue

alex-frankel avatar Jan 26 '22 20:01 alex-frankel

Anyway, back to the original proposal - using parameters as arguments for loadTextContent is not possible. Parameters are evaluated during the deployment (runtime), while file loading in loadTextContent is done during compilation (bicep build command). the azure cli does the compilation on the fly before the actual deployment occurs, hence the impression that it could be done.

The only way I can think of to provide this functionality is to introduce a compile-time parameters (similar to C# symbols - https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/preprocessor-directives#conditional-compilation). AFAIR there are some other issues that refers to introducing similar functionality (i.e. #3109, #1761). It might be worth to think about it. but IMHO it will not be as fast as you would like :)

miqm avatar Jan 27 '22 11:01 miqm

Hello, would be great if we would support this and make it more general. So it will also support similar functions like loadFileAsBase64. It would give more flexibility when working with modules.

cpinotossi avatar Feb 07 '22 17:02 cpinotossi

Would having function that loads json from file be a solution? Mind, that vars can be runtime values, while load* functions can accept only compile-time values as arguments.

miqm avatar Feb 07 '22 18:02 miqm

We have the same requirement\need, we are creating role definitions and looking to pass in the json file containing the permissions.

wsucoug69 avatar Apr 01 '22 14:04 wsucoug69

+1 to this. Have the same need for populating permissions from json.

garrettsingletary avatar Apr 01 '22 15:04 garrettsingletary

Hello @leechan-bicep , @alex-frankel

A workaround for this :

Hi, I have the same error when providing more than one file location getting the "The value must be a compile-time constant." error.Please let us know how to fix this issue

var stringArray = [ './policies/policy_1.json' ]

var policies = [for i in range(0, length(stringArray)): json(loadTextContent(stringArray[i]))]

May be : var stringArray = [ loadTextContent('./policy-1.json') loadTextContent('./policy-2.json') loadTextContent('./policy-3.json') ]

var policies = [for i in range(0, length(stringArray)): json(stringArray[i])] Since in anyways, we should put a constant path.

charotAmine avatar May 10 '22 12:05 charotAmine

hey, I tried @charotAmine code, but it didn't worked for me. But I created a similar way to solve this: @alex-frankel goes in the same direction like your code snippet

var stringArray = [
  json(loadTextContent('./file1.json'))
  json(loadTextContent('./file2json'))
  json(loadTextContent('./file3.json'))
  ]

module m_pol_def 'br/modules:microsoft.authorization.policydefinitions:1.0' = [for onePol in stringArray: { 
  name: onePol.name
    params: {...}
}]

SeSeicht avatar Jun 01 '22 17:06 SeSeicht

+1 for using this to load local files for openapi specs

calebherbison avatar Jun 20 '22 04:06 calebherbison

@alex-frankel

I have a similar situation which appears to be a cross between this issue and #3607.

In my case, I am looking to load multiple "resource configuration" files. These files may vary from deployment to deployment but will be re-usable. Depending on the specific scenario, I would like to point my Bicep configuration at a single JSON file containing a list of required "resource configuration" files, and then loop over that to load in the "resource configuration" files.

I will always know the location of the first file, and the point of this is to abstract the list of other sources from the template to simplify code maintenance and D.R.Y.

Take an example such as the following:

configLocations.Scenario1.json example

[
    "config/resource1settings.json",
    "config/resource2settings.json",
    "config/resource3settings.json",
    "config/resource4settings.json",
    "config/resource5settings.json",
    "config/resource6settings.json"
]

config/resource1settings.json example

{
  "setting1": "value",
  "setting2": 100,
  "setting3": true,
  "setting4": []
}

sample.bicep example

// The following var is used to load the list of resource configs to import
var listResourceConfigs = loadJsonContent('configLocations.Scenario1.json')

// The following var is used to load the resource configs
var resourceConfigs = [for config in listResourceConfigs: loadJsonContent(config)]

This results in the error The value must be a compile-time constant.bicep(BCP032):

Bicep error: The value must be a compile-time constant.bicep(BCP032)

As such, it would be really useful to have a way to do the following for compile-time operations:

  1. Declare constants (only visible in Bicep, maybe using a decorator format such as @constant(key='value'))
  2. Support for loops (again, possibly using a different format to declare the loops so they are explicitly identified as used at compile-time)

In my scenario, explicitly declaring the list of "resource configuration" files into the bicep file as proposed in the above "work-arounds" would defeat the point of this abstraction.

Thank you

krowlandson avatar Jul 21 '22 09:07 krowlandson

I have a similar requirement:

I try to configure an Azure Container App setting a pram to set the Resources CPU&RAM. I hit The value must be a compile-time constant.bicep(BCP032):

How to implement this please ?



@allowed([
  json('0.25')
  json('0.5')
  json('0.75')
  json('1.0')  
  json('1.25')
  json('1.5')
  json('1.75')
  json('2.0')    
])
@description('The container Resources CPU')
param containerResourcesCpu object = json('0.25')

@allowed([
  json('0.5')
  json('1.0')  
  json('1.5')
  json('2.0')    
  json('2.5')
  json('3.0')  
  json('3.5')
  json('4.0')    
])
@description('The container Resources Memory')
param containerResourcesMemory object = json('0.5')

resource ContainerApp 'Microsoft.App/containerApps@2022-03-01' = {
  properties: {
    template: {
      containers: [
        {
          resources: {
            cpu: containerResourcesCpu // json('0.5') //500m
            memory: containerResourcesMemory // Gi
          }

ezYakaEagle442 avatar Sep 26 '22 14:09 ezYakaEagle442

I have a similar requirement:

I try to configure an Azure Container App setting a pram to set the Resources CPU&RAM. I hit The value must be a compile-time constant.bicep(BCP032):

How to implement this please ?



@allowed([
  json('0.25')
  json('0.5')
  json('0.75')
  json('1.0')  
  json('1.25')
  json('1.5')
  json('1.75')
  json('2.0')    
])
@description('The container Resources CPU')
param containerResourcesCpu object = json('0.25')

@allowed([
  json('0.5')
  json('1.0')  
  json('1.5')
  json('2.0')    
  json('2.5')
  json('3.0')  
  json('3.5')
  json('4.0')    
])
@description('The container Resources Memory')
param containerResourcesMemory object = json('0.5')

resource ContainerApp 'Microsoft.App/containerApps@2022-03-01' = {
  properties: {
    template: {
      containers: [
        {
          resources: {
            cpu: containerResourcesCpu // json('0.5') //500m
            memory: containerResourcesMemory // Gi
          }

Your problem is related to lack of floating point number support in bicep: #1386

miqm avatar Sep 26 '22 16:09 miqm

Indeed, it is now fixed with :

@allowed([
  '0.25'
  '0.5'
  '0.75'
  '1.0' 
  '1.25'
  '1.5'
  '1.75'
  '2.0'    
])
@description('The container Resources CPU')
param containerResourcesCpu string = '0.5'

@allowed([
  '0.5'
  '1.0'  
  '1.5'
  '2.0'    
  '2.5'
  '3.0'  
  '3.5'
  '4.0'    
])
@description('The container Resources Memory')
param containerResourcesMemory string = '1.0'

resource ContainerApp 'Microsoft.App/containerApps@2022-03-01' = {
  properties: {
    template: {
      containers: [
        {
          resources: {
            cpu: json(containerResourcesCpu) // 250m
            memory: json(containerResourcesMemory) // Gi
          }

ezYakaEagle442 avatar Sep 26 '22 16:09 ezYakaEagle442

I believe this is a similar problem and wanted to see if there is a resolution yet. I'm working on generalizing deployment of VMs with BICEP templating. One of the things I want to do is manage which machine sizes the deployment has access to, but also control for different deployment configuration for GPU machines vs CPU machines. Thus, I'd like to do:

var allowed_cpu = array([
  'standard_b1ms'
])
var allowed_gpu = array([
  'Standard_NC6s_v3'
  'Standard_NC12s_v3'
  'Standard_NC4as_T4_v3'
])
var allowed_vms = concat(allowed_cpu, allowed_gpu)
@allowed(allowed_vms)
param vm_type string = 'standard_b1ms'

However, doing so marks the @allowed attempt as "must be a compile time constant". Is it still impossible to uses var as part of @allowed?

ZWMiller avatar Oct 03 '22 18:10 ZWMiller

I believe this is a similar problem and wanted to see if there is a resolution yet. I'm working on generalizing deployment of VMs with BICEP templating. One of the things I want to do is manage which machine sizes the deployment has access to, but also control for different deployment configuration for GPU machines vs CPU machines. Thus, I'd like to do:

var allowed_cpu = array([
  'standard_b1ms'
])
var allowed_gpu = array([
  'Standard_NC6s_v3'
  'Standard_NC12s_v3'
  'Standard_NC4as_T4_v3'
])
var allowed_vms = concat(allowed_cpu, allowed_gpu)
@allowed(allowed_vms)
param vm_type string = 'standard_b1ms'

However, doing so marks the @allowed attempt as "must be a compile time constant". Is it still impossible to uses var as part of @allowed?

var yes, but not concat - as this is a runtime function.

However you should be able to use loadJsonContent inside @allowed and keep VM list in separate json and use query parameter to get what you need - but I see it does not work :/ I'll put an issue and try to fix it.

miqm avatar Oct 04 '22 13:10 miqm

The runtime engine doesn't support variable() function expressions within an allowedValues parameter property, so we would either need to update the engine to support such expressions or convert the result of loadJsonContent to a string literal rather than a template expression.

image image

jeskew avatar Oct 05 '22 17:10 jeskew

+1 to the request.

elenalash avatar Dec 12 '22 04:12 elenalash

@alex-frankel +1. This is a frequent use case for us.

akhilthomas011 avatar Jan 05 '23 15:01 akhilthomas011

+1

Kaloszer avatar Jan 18 '23 11:01 Kaloszer

+1

sofia-valles avatar Jan 23 '23 10:01 sofia-valles

+1

Gabriel123N avatar Mar 08 '23 09:03 Gabriel123N

+1

tuomaskujala avatar Mar 14 '23 10:03 tuomaskujala