Multiple Resource Group Support --> Unable to deploy Azure Function app

[mcollier]

Output from az dev version azd version 0.0.1-beta.1588565 (commit 1b0d7eba856333b956b09e929666838dbf2df5e9)

Describe the bug Unable to deploy code to Azure Function app.

To Reproduce

• Execute azd init on existing project.
• Add content to the azure.yaml file

  name: gridwich-ocw22
  services:
  fn:
     project: src/Gridwich.Host.FunctionApp/src
     resourceName: gridwichfxn${GRIDWICH_ENVIRONMENT}
     language: csharp
     host: function
  

• Azure Function code is located at C:\source\msft\gridwich-ocw22\src\Gridwich.Host.FunctionApp.
• Execute azd deploy --debug from C:\source\msft\gridwich-ocw22 via a Visual Studio Code terminal.

Expected behavior Azure Dev CLI should successfully deploy code to a provisioned Azure Function app. If the deployment is unsuccessful, the Azure Dev CLI should return an actionable error message (clear error message, suggestions to resolve, suggestions to learn more, etc.)

Environment Information on your environment: * .NET version: 3.1.417 * Windows: Version 22H2 (OS Build 22621.1) * PowerShell: 7.2.4 * Azure CLI: 2.36.0

Additional context

Running the azd deploy --debug command produces the following output:

2022/05/20 14:17:39 executil.go:235: Cmd: az account list --output json --query [].{name:name, id:id, isDefault:isDefault} --debug
Out:[
  {
   // List of Azure subscriptions removed.
  }
]

Err:DEBUG: cli.knack.cli: Command arguments: ['account', 'list', '--output', 'json', '--query', '[].{name:name, id:id, isDefault:isDefault}', '--debug']
DEBUG: cli.knack.cli: __init__ debug log:
Cannot enable color.
DEBUG: cli.knack.cli: Event: Cli.PreExecute []
DEBUG: cli.knack.cli: Event: CommandParser.OnGlobalArgumentsCreate [<function CLILogging.on_global_arguments at 0x03D28CD0>, <function OutputProducer.on_global_arguments at 0x03EED610>, <function CLIQuery.on_global_arguments at 0x03F0B268>]
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPreCommandTableCreate []
DEBUG: cli.azure.cli.core: Modules found from index for 'account': ['azure.cli.command_modules.profile', 'azure.cli.command_modules.resource']
DEBUG: cli.azure.cli.core: Loading command modules:
DEBUG: cli.azure.cli.core: Name                  Load Time    Groups  Commands
DEBUG: cli.azure.cli.core: profile                   0.007         2         9
DEBUG: cli.azure.cli.core: resource                  0.012        43       197
DEBUG: cli.azure.cli.core: Total (2)                 0.019        45       206
DEBUG: cli.azure.cli.core: Loaded 44 groups, 206 commands.
DEBUG: cli.azure.cli.core: Found a match in the command table.
DEBUG: cli.azure.cli.core: Raw command  : account list
DEBUG: cli.azure.cli.core: Command table: account list
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPreCommandTableTruncate [<function AzCliLogging.init_command_file_logging at 0x0499C190>]
DEBUG: cli.azure.cli.core.azlogging: metadata file logging enabled - writing logs to 'C:\Users\mcollier\.azure\commands\2022-05-20.14-17-39.account_list.62144.log'.
INFO: az_command_data_logger: command args: account list --output {} --query {} --debug
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPreArgumentLoad [<function register_global_subscription_argument.<locals>.add_subscription_parameter at 0x049D3F10>]
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPostArgumentLoad []
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPostCommandTableCreate [<function register_ids_argument.<locals>.add_ids_arguments at 0x049E3028>, <function register_cache_arguments.<locals>.add_cache_arguments at 0x049EB4F0>]
DEBUG: cli.knack.cli: Event: CommandInvoker.OnCommandTableLoaded []
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPreParseArgs []
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPostParseArgs [<function OutputProducer.handle_output_argument at 0x03EED658>, <function CLIQuery.handle_query_parameter at 0x03F0B2B0>, <function register_ids_argument.<locals>.parse_ids_arguments at 0x049EB538>]
DEBUG: cli.knack.cli: Event: CommandInvoker.OnTransformResult [<function _resource_group_transform at 0x049D31D8>, <function _x509_from_base64_to_hex_transform at 0x049D3220>]
DEBUG: cli.knack.cli: Event: CommandInvoker.OnFilterResult [<function CLIQuery.handle_query_parameter.<locals>.filter_output at 0x04AE45C8>]
DEBUG: cli.knack.cli: Event: Cli.SuccessfulExecute []
DEBUG: cli.knack.cli: Event: Cli.PostExecute [<function AzCliLogging.deinit_cmd_metadata_logging at 0x0499C2B0>]
INFO: az_command_data_logger: exit code: 0
INFO: cli.__main__: Command ran in 0.841 seconds (init: 0.766, invoke: 0.075)
INFO: telemetry.save: Save telemetry record of length 2929 in cache
INFO: telemetry.check: Returns Positive.
INFO: telemetry.main: Begin creating telemetry upload process.
INFO: telemetry.process: Creating upload process: "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\python.exe C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\Lib\site-packages\azure\cli\telemetry\__init__.pyc C:\Users\mcollier\.azure"
INFO: telemetry.process: Return from creating process
INFO: telemetry.main: Finish creating telemetry upload process.

2022/05/20 14:17:39 service.go:33: packing service fn
Deploying service fn -2022/05/20 14:18:45 service.go:40: deploying service fn
Deployed service fn
Error: Command Failed: deploying service fn: deploying service fn package: exit status 3

[richardpark-msft]

I'm taking a look at this. It looks like we have code to dump out the stdout/stderr of the failed command but it doesn't appear to have done anything here.

[richardpark-msft]

@mcollier : I made it so failures should print the actual failures from the deploy: Azure/azure-dev-pr#1128

Do you mind grabbing the latest azd build and trying this again? We do have tests for function apps that run regularly, so the basics should be working but there might be something we've missed.

[mcollier]

@richardpark-msft I hope to get time to try later today. :)

[mcollier]

The azd deploy --debug command certainly prints a lot more information now!

The extra info helped me to get an idea of what is likely failing. I noticed a "Resource group 'gwocwrg' could not be found" error message in the output.

The Dev CLI assumes the function is in the resource group named ${AZURE_ENV_NAME}rg. I noticed this Azure CLI command in the debug output: az functionapp deployment source config-zip --subscription xxxxxx --resource-group gwocwrg --name gridwichfxnsb --src C:\Users\mcollier\AppData\Local\Temp\azddeploy446653092.zip --build-remote true --timeout 3600 --debug

However, the target Function app isn't in that resource group.

Is there a way to tell the Az Dev CLI to deploy a function to a specific Function app in a specific resource group? Or, to put another way, is there a way to specify the target function app? I think what's happening is that my main.bicep creates multiple resource groups, each with various resources.

[ellismg]

Is there a way to tell the Az Dev CLI to deploy a function to a specific Function app in a specific resource group? Or, to put another way, is there a way to specify the target function app? I think what's happening is that my main.bicep creates multiple resource groups, each with various resources.

Ahh yes - that is very likely the problem. Thanks for digging in @mcollier! This is more or less Azure/azure-dev-issue-transfer#133, which we don't have an answer for today.

I think it would be quick for us to add a resourceGroup parameter to each service for azure.yaml that would allow you to control this. Longer team, we may be able to make this smarter by looking at the resources for your deployment and doing a match just by app service name. But that's a bit larger undertaking, and I suspect we would want a way to set this explicitly as well.

[jongio]

Triage: Not a blocker, but Matt thinks he can do this for public preview.

[KSchlobohm]

I've been building a project with azd that includes a production ready environment and found that high availability is another scenario that uses multiple resource groups. In that scenario I want my resources.bicep to include the ability to choose production ready SKUs when targeting production and dev resources when deploying to non-prod. As part of the production deployment, I also want to target multiple regions to achieve a higher composite SLA.

Having these requirements makes it harder to use 1 bicep template and 1 azure.yaml to deploy these resources but it's not impossible. To achieve the prod sku selection I can use azd env set IS_PROD true and bicep conditional deployments. But, this decision also impacts the azd deploy and azd up behavior because I need to help azd understand that I want to deploy the same web app twice. Based on the error message there are two options:

1. Specify the resource group in azure.yaml - my resource group is still named by the user's choice of environment, so I won't be able to hard code this into the azure.yaml
2. Set the AZURE_RESOURCE_GROUP environment variable - this works for my scenario as the user was already calling azd env set IS_PROD to toggle the new behavior in the template. But, I still need to deploy twice and cannot reduce the operation down to azd up.

It would be nice to have a way to simplify this so I can do a clean checkout and then run azd up. Maybe tags could work as a way to target the correct resource group by service. And, since I'm deploying the same web app to two regions, it would be nice to be able to associate a service with multiple tags so I can simplify back to azd up.

[karolz-ms]

@KSchlobohm in your scenario, maybe I am not entirely understanding your requirements, but it seems that once you configure the production and development environments, the command

azd up --environment production && azd up --environment development

should get the two instances of your app deployed in a way that you want? Or am I missing something?

[KSchlobohm]

I was previously using the environment name to trigger the conditional bicep behavior as you're describing but I started preferring parameters to control, and describe, the outcome vs the magic string behavior achieved by if (startsWith(environment, 'prod') || endsWith(environment, 'prod')). This approach started when I noticed that the resourceGroup name would be "${environment}-rg" and I decided that I wanted the resource group name to describe the solution (e.g. eshopprod is closer to what my environment name looked like).

The problem with associating a web app (a service in my azure.yaml) with multiple resource groups still remains because the environment is only multiregional for production.

In this scenario my bicep templates default to the non-prod, single region, deployment. So when I deploy for high availability, I get an error from azd deploy because I only have 1 service described in my azure.yaml file but I want that service deployed to two places. If I changed azure.yaml to describe the same service deployed to two resource groups, then that would not align with the budget plan for the non-prod environments.

ps - I may not be aware of a feature I can use in my azure.yaml to describe the correct resource group based on the deployed resources. if there is a parameter, or a conditional, I can use to describe the target resource group pls let me know

[karolz-ms]

Thank you for clarification!

I am not aware in anything in azd today that lets one specify different resource groups for different resources that are part of the application. I think Matt mentioned it would be an enhancement here https://github.com/Azure/azure-dev/issues/328#issuecomment-1205792956

That said, question @KSchlobohm : in your "production" scenario, do the two instances of App Service have to be in different resource groups? Or can they be in the same resource group, but just in different regions? Because if it is the latter, that might be easier to do without changing azure.yaml format. We would just need to make sure that when azd deploys a service, it deploys it to all targets (target being AppService app plan in this case) tagged with corresponding service name.

[KSchlobohm]

Deploying to two azure locations within the same resource group works for my scenario. Thanks @karolz-ms

[ellismg]

It would be nice to have a way to simplify this so I can do a clean checkout and then run azd up. Maybe tags could work as a way to target the correct resource group by service. And, since I'm deploying the same web app to two regions, it would be nice to be able to associate a service with multiple tags so I can simplify back to azd up.

We have been exploring using tags in this way. In the tool today we have support for finding a target resource based on a tag. If you tag the resource for your service with azd-service-name and then the "friendly name" of your service from azure.yaml (that's the name you use as the key under the services section, e.g. fn, in the example above) then the tool should try to use it (we added this in part to support cases like yours where the resource name will be unguessable).

Looking at the code however, it seems like this wouldn't work for you out of the box because we restrict our query to a resource group, but I'm not 100% sure why we decided to that that, possibly performence?

https://github.com/Azure/azure-dev/blob/6792d923cf09c11c8733c7d5fd385d030372b877/cli/azd/pkg/project/service.go#L96-L104

@wbreza Do you have thoughts here? I wonder if removing the restriction on resource groups in the query would be sufficient? As an FYI - In the environment proposal I am working on I have added an id value to every environment, so you could imagine having a tag like azd-env-id and using that (combined with `azd-service-name) instead of a fixed resource group.

[jongio]

@ellismg - If you don't filter by RG you'll get resources that have the same azd-service-name tag in two diff rgs and results can be unpredictable.

[jongio]

@mcollier are you okay closing this issue in favor of: https://github.com/Azure/azure-dev/issues/690

[mcollier]

@jongio Yes.