copyIndex error "The provided copy name '' doesn't exist in the resource" when using triple FOR-loops for the same array

[doubleyouvdb]

Bicep version 0.4.63

Describe the bug When you are using in a variable with FOR-loop a copyIndex to refer to a certain child resource reference in another FOR-loop for the same array of objects which also uses an index to retrieve its parent that is also in another FOR-loop for the same set of objects, then in the autogenerated ARM template, all the translated copyIndexes for the parent in the copy variable get blanked out.

To Reproduce Deploy AKS Cluster with the following code for agent pool profiles. For example purpose, only 'name' and the here relevant property 'vnetSubnetID' are listed. The properties of both the resource references and of the variable 'agentPoolProfiles' are all populated using the same array [for agentPool in aksCluster.agentPools].

// Get existing vnets
resource virtualNetworks 'Microsoft.Network/virtualNetworks@2021-02-01' existing = [for agentPool in aksCluster.agentPools: {
  name: union(defaultAgentPoolObject, agentPool).virtualNetwork.virtualNetworkName
  scope: resourceGroup(union(defaultAgentPoolObject, agentPool).virtualNetwork.subscriptionId, union(defaultAgentPoolObject, agentPool).virtualNetwork.resourceGroup)
}]

// Get existing subnets
resource subnets 'Microsoft.Network/virtualNetworks/subnets@2021-02-01' existing = [for (agentPool, i) in aksCluster.agentPools: {
  name: union(defaultAgentPoolObject, agentPool).virtualNetwork.subnetName
  parent: virtualNetworks[i]
}]

var agentPoolProfiles = [for (agentPool, i) in aksCluster.agentPools: {
  name: agentPool.name
  vnetSubnetID: aksCluster.network.plugin =~ 'azure' ? subnets[i].id : null
}]

When deploying, this generates error:

An error occurred while deploying this offering. Error: 
InvalidTemplate - Long running operation failed with status
'Failed'. Additional Info:'Deployment template language
expression evaluation failed: 'The template language function
'copyIndex' has an invalid argument. The provided copy name ''
doesn't exist in the resource. Please see
https://aka.ms/arm-copy for usage details.'. Please see
https://aka.ms/arm-template-expressions for usage details.'

When examining the autogenerated ARM template, the cause is clear: in the variable, the copyIndex() where a property of the parent resource is called, is always empty. It should be copyIndex('agentPoolProfiles').

"copy": [
      {
        "name": "agentPoolProfiles",
        "count": "[length(parameters('aksCluster').agentPools)]",
        "input": {
          "name": "[parameters('aksCluster').agentPools[copyIndex('agentPoolProfiles')].name]",
          "vnetSubnetID": "[if(equals(toLower(parameters('aksCluster').network.plugin), toLower('azure')), extensionResourceId(format('/subscriptions/{0}/resourceGroups/{1}', union(variables('defaultAgentPoolObject'), parameters('aksCluster').agentPools[copyIndex()]).virtualNetwork.subscriptionId, union(variables('defaultAgentPoolObject'), parameters('aksCluster').agentPools[copyIndex()]).virtualNetwork.resourceGroup), 'Microsoft.Network/virtualNetworks/subnets', union(variables('defaultAgentPoolObject'), parameters('aksCluster').agentPools[copyIndex()]).virtualNetwork.virtualNetworkName, union(variables('defaultAgentPoolObject'), parameters('aksCluster').agentPools[copyIndex('agentPoolProfiles')]).virtualNetwork.subnetName), null())]"
        }

Additional context N/A

[doubleyouvdb]

Btw, found the following workaround to be valid:

// Get existing subnets
resource subnets 'Microsoft.Network/virtualNetworks/subnets@2021-02-01' existing = [for agentPool in aksCluster.agentPools: if (aksCluster.network.plugin =~ 'azure') {
  name: '${agentPool.virtualNetwork.virtualNetworkName}/${agentPool.virtualNetwork.subnetName}'
}]

[slavizh]

+1

[brwilkinson]

here is the simple (standalone) repro for this... removing any complexity of the agentPools or AKS.

• Looping through VNETS and then looping through Subnets while referencing the VNET (loop) for the parent.

var agentPools = [
  {
    name: 'NodePool1'
    virtualNetwork: {
      virtualNetworkName: 'vnet1'
      subnetName: 'subnet1'
    }
  }
]

resource virtualNetworks 'Microsoft.Network/virtualNetworks@2021-02-01' existing = [for agentPool in agentPools: {
  name: agentPool.virtualNetwork.virtualNetworkName
}]

resource subnets 'Microsoft.Network/virtualNetworks/subnets@2021-02-01' existing = [for (agentPool, index) in agentPools: {
  name: agentPool.virtualNetwork.subnetName
  parent: virtualNetworks[index]
}]

var agentPoolProfiles = [for (agentPool, index) in agentPools: {
  name: agentPool.name
  vnetSubnetID: subnets[index].id
}]

output agentPoolProfiles array = agentPoolProfiles

Expected compiled:

"vnetSubnetID": "[resourceId('Microsoft.Network/virtualNetworks/subnets', variables('agentPools')[copyIndex('agentPoolProfiles')].virtualNetwork.virtualNetworkName, variables('agentPools')[copyIndex('agentPoolProfiles')].virtualNetwork.subnetName)]"

Actually compiled:

"vnetSubnetID": "[resourceId('Microsoft.Network/virtualNetworks/subnets', variables('agentPools')[copyIndex()].virtualNetwork.virtualNetworkName, variables('agentPools')[copyIndex('agentPoolProfiles')].virtualNetwork.subnetName)]"

[majastrz]

Thanks for the self-contained repro @brwilkinson. It looks like the root cause is that we're not doing index replacement in a way that preserves the local loop name.

[santo2]

what is the status of this? I'm facing this issue as well.

[slavizh]

very annoying bug that for me shouldn't be hard to fix. It quite common among many resources as in many cases you have to reference subnet IDs

[santo2]

@alex-frankel I saw the mention on the community call as well, that's already good news ;-) Thanks

[santo2]

this seems to keep moving to vnext, any update on this?

[majastrz]

I'm currently investigating this issue. There's a deeper problem with index expression substitution, which makes this more complicated to fix unfortunately.

[santo2]

could we possibly get an update on this during the community call? it's quite an essential part of using references to another resource which is now not working for certain resources in our scripts. we are awaiting a fix to be able to use bicep for some of our script deployments.

• this issue is almost a year old now

[majastrz]

We are working on a new JSON format that exposes the concepts of symbolic names and existing resources in the JSON. You can enable it in Bicep by setting the BICEP_SYMBOLIC_NAME_CODEGEN_EXPERIMENTAL environment variable to true. For best experience, should be enabled for both Bicep CLI and VS code. One of the goals of the new JSON format is to simplify the code generation of Bicep and eliminate bugs like this one.

I tested it with the @brwilkinson's template and it seemed to produce correct JSON, but I haven't tried deploying it. Can you try turning in on and see if it helps?

[slavizh]

@majastrz can you point me to the document how to set environment variable. I have not done this before and I have not found such reference on internet.

[majastrz]

We don't have any docs that cover this specifically for Bicep. These are just normal system environment variables. Depending on how you're invoking Bicep, the following options may work for you. In a Windows .bat or .cmd file:

set BICEP_SYMBOLIC_NAME_CODEGEN_EXPERIMENTAL=true

In PowerShell:

$env:BICEP_SYMBOLIC_NAME_CODEGEN_EXPERIMENTAL = "true"

In GitHub actions or ADO yaml:

env:
  BICEP_SYMBOLIC_NAME_CODEGEN_EXPERIMENTAL: true

Overtime, we will be adding more experimental features that can be enabled. To make it easier to use them, @jeskew will expose settings to turn on experimental Bicep features in .bicepconfig, but it's not available yet.

For best experience, you should set the env var before opening VS code and before running any Bicep CLI commands. This way the experience will be consistent.

[slavizh]

@majastrz thanks for the help. I was suspecting that might be the way but I wanted to be sure. I have managed to enable the setting but unfortunately without result. Bellow error is generated.

WARNING: Symbolic name support in ARM is experimental, and should be enabled for testing purposes only. Do not enable this setting for any production usage, or you may be unexpectedly broken at any time!
D:\dev\Sentia\lz\lz-kubernetes\test.bicep(25,5) : Warning no-unused-vars: Variable "agentPoolProfiles" is declared but never used. [https://aka.ms/bicep/linter/no-unused-vars]
Unhandled exception. System.NotImplementedException: Mismatch between count of index expressions and inaccessible symbols during array access index replacement.
   at Bicep.Core.Emit.ExpressionConverter.CreateConverterForIndexReplacement(SyntaxBase nameSyntax, SyntaxBase indexExpression, SyntaxBase newContext) in C:\__w\1\s\bicep\src\Bicep.Core\Emit\ExpressionConverter.cs:line 207
   at Bicep.Core.Emit.ExpressionEmitter.EmitExpression(SyntaxBase resourceNameSyntax, SyntaxBase indexExpression, SyntaxBase newContext) in C:\__w\1\s\bicep\src\Bicep.Core\Emit\ExpressionEmitter.cs:line 98
   at Bicep.Core.Emit.ScopeHelper.<>c__DisplayClass7_0.<EmitResourceOrModuleScopeProperties>b__0() in C:\__w\1\s\bicep\src\Bicep.Core\Emit\ScopeHelper.cs:line 367
   at Bicep.Core.Emit.PositionTrackingJsonTextWriter.WritePropertyWithPosition(SyntaxBase keyPosition, String name, Action valueFunc) in C:\__w\1\s\bicep\src\Bicep.Core\Emit\PositionTrackingJsonTextWriter.cs:line 108
   at Bicep.Core.Emit.ExpressionEmitter.EmitPropertyInternal(LanguageExpression expressionKey, Action valueFunc, SyntaxBase location, Boolean skipCopyCheck) in C:\__w\1\s\bicep\src\Bicep.Core\Emit\ExpressionEmitter.cs:line 451
   at Bicep.Core.Emit.ExpressionEmitter.EmitProperty(String name, Action valueFunc) in C:\__w\1\s\bicep\src\Bicep.Core\Emit\ExpressionEmitter.cs:line 429
   at Bicep.Core.Emit.ScopeHelper.EmitResourceOrModuleScopeProperties(SemanticModel semanticModel, ScopeData scopeData, ExpressionEmitter expressionEmitter, SyntaxBase newContext) in C:\__w\1\s\bicep\src\Bicep.Core\Emit\ScopeHelper.cs:line 367
   at Bicep.Core.Emit.ScopeHelper.EmitResourceScopeProperties(SemanticModel semanticModel, ScopeData scopeData, ExpressionEmitter expressionEmitter, SyntaxBase newContext) in C:\__w\1\s\bicep\src\Bicep.Core\Emit\ScopeHelper.cs:line 327
   at Bicep.Core.Emit.TemplateWriter.<>c__DisplayClass21_0.<EmitResource>b__0() in C:\__w\1\s\bicep\src\Bicep.Core\Emit\TemplateWriter.cs:line 509
   at Bicep.Core.Emit.PositionTrackingJsonTextWriter.WriteObjectWithPosition(SyntaxBase sourcePosition, Action propertiesFunc) in C:\__w\1\s\bicep\src\Bicep.Core\Emit\PositionTrackingJsonTextWriter.cs:line 97
   at Bicep.Core.Emit.TemplateWriter.EmitResource(PositionTrackingJsonTextWriter jsonWriter, DeclaredResourceMetadata resource, ExpressionEmitter emitter) in C:\__w\1\s\bicep\src\Bicep.Core\Emit\TemplateWriter.cs:line 400
   at Bicep.Core.Emit.TemplateWriter.EmitResources(PositionTrackingJsonTextWriter jsonWriter, ExpressionEmitter emitter) in C:\__w\1\s\bicep\src\Bicep.Core\Emit\TemplateWriter.cs:line 358
   at Bicep.Core.Emit.TemplateWriter.GenerateTemplateWithoutHash(PositionTrackingJsonTextWriter jsonWriter) in C:\__w\1\s\bicep\src\Bicep.Core\Emit\TemplateWriter.cs:line 137
   at Bicep.Core.Emit.TemplateWriter.Write(SourceAwareJsonTextWriter writer) in C:\__w\1\s\bicep\src\Bicep.Core\Emit\TemplateWriter.cs:line 103
   at Bicep.Core.Emit.TemplateEmitter.<>c__DisplayClass8_0.<Emit>b__0() in C:\__w\1\s\bicep\src\Bicep.Core\Emit\TemplateEmitter.cs:line 68
   at Bicep.Core.Emit.TemplateEmitter.EmitOrFail(Func`1 write) in C:\__w\1\s\bicep\src\Bicep.Core\Emit\TemplateEmitter.cs:line 123
   at Bicep.Core.Emit.TemplateEmitter.Emit(Stream stream) in C:\__w\1\s\bicep\src\Bicep.Core\Emit\TemplateEmitter.cs:line 59
   at Bicep.Cli.Services.CompilationWriter.ToFile(Compilation compilation, String outputPath) in C:\__w\1\s\bicep\src\Bicep.Cli\Services\CompilationWriter.cs:line 27
   at Bicep.Cli.Commands.BuildCommand.RunAsync(BuildArguments args) in C:\__w\1\s\bicep\src\Bicep.Cli\Commands\BuildCommand.cs:line 69
   at Bicep.Cli.Program.RunAsync(String[] args) in C:\__w\1\s\bicep\src\Bicep.Cli\Program.cs:line 78
   at Bicep.Cli.Program.Main(String[] args) in C:\__w\1\s\bicep\src\Bicep.Cli\Program.cs:line 65
   at Bicep.Cli.Program.<Main>(String[] args)

The minimum code to reproduce it:

param aksCluster object

var defaultAgentPool = {
  virtualNetwork: {
    subscriptionId: subscription().subscriptionId
    resourceGroup: 'dummy'
    virtualNetworkName: ''
    subnetName: ''
  }
}

resource virtualNetworks 'Microsoft.Network/virtualNetworks@2021-08-01' existing = [for agentPool in aksCluster.agentPools: {
  name: union(defaultAgentPool, agentPool).virtualNetwork.virtualNetworkName
  scope: resourceGroup(union(defaultAgentPool, agentPool).virtualNetwork.subscriptionId, agentPool.virtualNetwork.resourceGroup)
}]

// // Get existing subnets
resource subnets 'Microsoft.Network/virtualNetworks/subnets@2021-08-01' existing = [for (agentPool, i) in aksCluster.agentPools: {
  name: union(defaultAgentPool, agentPool).virtualNetwork.subnetName
  parent: virtualNetworks[i]
}]


var agentPoolProfiles = [for (agentPool, i) in aksCluster.agentPools: {
  // only agentPoolProfiles has name as property, agentPools do not have it
  nameProperty: {
    name: agentPool.name
  }
  commonProperties: {
    vnetSubnetID: empty(union(defaultAgentPool, agentPool).virtualNetwork.subnetName) ? null : subnets[i].id
  }
}]

[slavizh]

@majastrz have you managed to look why the new method gives error on this case?

[majastrz]

Yes, we're not picking up the index expression from the parent property which fails an assertion in loop code generation. I'm working on the fix.

[Kaloszer]

+1

[slavizh]

@majastrz was anything done in recent versions of Bicep to fix this? When I use the latest sample for reproduction and generate the ARM template it seems that copyIndex() has parameter now so it seems fixed. I am yet to test it with actual deployment but if the ARM code is correct there should not be a problem.

[majastrz]

I think @jeskew made a change that fixed loop codegen with indexed scopes. I will double check.

[slavizh]

I have done some tests and they show that it is fixed now.

[majastrz]

Thanks for confirming. I will also add your repro as a test case, so it doesn't regress in the future.

[alex-frankel]

Closing since it appears this is resolved.

[jupacaza]

Hello, I am reproducing a very issue today with Bicep CLI version 0.28.1 Bicep CLI version 0.28.1 (ba1e9f8c1e)

I define variables with the objects for the private endpoints that the AVM modules require. The "subnetResourceId" property adequately takes the copyIndex('storageAccountPrivateEndpoints') value but the privateDnsZoneResourceIds only says copyIndex() so I get "The provided copy name '' doesn't exist in the resource" error when deploying.

Attaching reproduction files. simple.bicep.txt simple.json.txt

Bicep:

var storageAccountPrivateEndpoints = [
  for (config, i) in stampConfigs: {
    subnetResourceId: privateEndpointSubnets[i].id
    location: config.location
    service: 'blob'
    privateDnsZoneGroupName: 'privatelink.blob.${environment().suffixes.storage}'
    privateDnsZoneResourceIds: [
      storageBlobPrivateDnsZones[i].id
    ]
  }
]

Expected:

"variables": {
    "copy": [
      {
        "name": "storageAccountPrivateEndpoints",
        "count": "[length(parameters('stampConfigs'))]",
        "input": {
          "subnetResourceId": "[extensionResourceId(format('/subscriptions/{0}/resourceGroups/{1}', subscription().subscriptionId, format('rg-{0}-{1}-{2}', parameters('networkWorkloadName'), parameters('stampConfigs')[copyIndex('storageAccountPrivateEndpoints')].location, parameters('stampConfigs')[copyIndex('storageAccountPrivateEndpoints')].stamp)), 'Microsoft.Network/virtualNetworks/subnets', split(format('vnet-apps-{0}-{1}-{2}/{3}', parameters('networkWorkloadName'), parameters('stampConfigs')[copyIndex('storageAccountPrivateEndpoints')].shortLocation, parameters('stampConfigs')[copyIndex('storageAccountPrivateEndpoints')].stamp, variables('privateEndpointsSnetName')), '/')[0], split(format('vnet-apps-{0}-{1}-{2}/{3}', parameters('networkWorkloadName'), parameters('stampConfigs')[copyIndex('storageAccountPrivateEndpoints')].shortLocation, parameters('stampConfigs')[copyIndex('storageAccountPrivateEndpoints')].stamp, variables('privateEndpointsSnetName')), '/')[1])]",
          "location": "[parameters('stampConfigs')[copyIndex('storageAccountPrivateEndpoints')].location]",
          "service": "blob",
          "privateDnsZoneGroupName": "[format('privatelink.blob.{0}', environment().suffixes.storage)]",
          "privateDnsZoneResourceIds": [
            "[extensionResourceId(format('/subscriptions/{0}/resourceGroups/{1}', subscription().subscriptionId, format('rg-{0}-{1}-{2}', parameters('networkWorkloadName'), parameters('stampConfigs')[copyIndex('storageAccountPrivateEndpoints')].location, parameters('stampConfigs')[copyIndex('storageAccountPrivateEndpoints')].stamp)), 'Microsoft.Network/privateDnsZones', format('privatelink.blob.{0}', environment().suffixes.storage))]"
          ]
        }
      }
    ],
...

Actual:

"variables": {
    "copy": [
      {
        "name": "storageAccountPrivateEndpoints",
        "count": "[length(parameters('stampConfigs'))]",
        "input": {
          "subnetResourceId": "[extensionResourceId(format('/subscriptions/{0}/resourceGroups/{1}', subscription().subscriptionId, format('rg-{0}-{1}-{2}', parameters('networkWorkloadName'), parameters('stampConfigs')[copyIndex('storageAccountPrivateEndpoints')].location, parameters('stampConfigs')[copyIndex('storageAccountPrivateEndpoints')].stamp)), 'Microsoft.Network/virtualNetworks/subnets', split(format('vnet-apps-{0}-{1}-{2}/{3}', parameters('networkWorkloadName'), parameters('stampConfigs')[copyIndex('storageAccountPrivateEndpoints')].shortLocation, parameters('stampConfigs')[copyIndex('storageAccountPrivateEndpoints')].stamp, variables('privateEndpointsSnetName')), '/')[0], split(format('vnet-apps-{0}-{1}-{2}/{3}', parameters('networkWorkloadName'), parameters('stampConfigs')[copyIndex('storageAccountPrivateEndpoints')].shortLocation, parameters('stampConfigs')[copyIndex('storageAccountPrivateEndpoints')].stamp, variables('privateEndpointsSnetName')), '/')[1])]",
          "location": "[parameters('stampConfigs')[copyIndex('storageAccountPrivateEndpoints')].location]",
          "service": "blob",
          "privateDnsZoneGroupName": "[format('privatelink.blob.{0}', environment().suffixes.storage)]",
          "privateDnsZoneResourceIds": [
            "[extensionResourceId(format('/subscriptions/{0}/resourceGroups/{1}', subscription().subscriptionId, format('rg-{0}-{1}-{2}', parameters('networkWorkloadName'), parameters('stampConfigs')[copyIndex()].location, parameters('stampConfigs')[copyIndex()].stamp)), 'Microsoft.Network/privateDnsZones', format('privatelink.blob.{0}', environment().suffixes.storage))]"
          ]
        }
      }
    ],

[jupacaza]

Update:

it seems my issue stemps from using fixed values for the private DNS zone names. The copyIndex('...') value is not populate when I do not reference the objects of the iterated array in the name.

This does not work:

resource storageBlobPrivateDnsZones 'Microsoft.Network/privateDnsZones@2020-06-01' existing = [
  for (config, i) in stampConfigs: {
    name: 'privatelink.blob.${environment().suffixes.storage}' // NOTE THE FIXED VALUE FOR ALL ITEMS IN ARRAY
    scope: resourceGroup('rg-${networkWorkloadName}-${config.location}-${config.stamp}')
  }
]

This works:

var networkResources = [for (config, i) in stampConfigs: {
  rg: 'rg-${networkWorkloadName}-${config.location}-${config.stamp}'
  storageBlobPrivateDnsZoneName: 'privatelink.blob.${environment().suffixes.storage}'
}]

resource storageBlobPrivateDnsZones 'Microsoft.Network/privateDnsZones@2020-06-01' existing = [
  for r in networkResources: {
    name: r.storageBlobPrivateDnsZoneName // NOTE NAME REFERENCES SOMETHING IN THE ITERATED ARRAY
    scope: resourceGroup(r.rg)
  }
]