`azd env set` in a hook script doesn't have the variables available in future steps

[aaronpowell]

• [x] Make sure you've installed the latest version using instructions in the wiki

Output from azd version Run azd version and copy and paste the output here:

azd version 1.9.3 (commit e1624330dcc7dde440ecc1eda06aac40e68aa0a3)

Describe the bug

If you use a hook, such as preprovision, to add environment variables to azd using azd env set, those variables aren't available until the azd session is restarted (meaning you can't use azd up).

We come across this in https://github.com/microsoft/azure-openai-service-proxy/ in which we have to create an Entra ID app registration and then want to push the client id/tenant id to services that azd will deploy.

Here is the preprovision hook definition and then here is the script.

The reason that we rely on this is there is no bicep support for deploying app registrations presently, so we use the Azure CLI to handle it.

Ideally, each step of the azd pipeline would load the .env file, even if it's being run through a group command like azd up.

[wbreza]

@aaronpowell Can you provide some more information on the issue you are facing?

During the execution of any hooks azd will automatically hoist the current environment variables from the azd environment and make them available in the hook script.

Conversely any calls to azd env set <key> <value> from within custom hooks will be immediately loaded back into the azd environment for use by azd or any downstream hooks.

Some feedback regarding your linked script. I noticed you are running the following:

# Use the `get-values` azd command to retrieve environment variables from the `.env` file
while IFS='=' read -r key value; do
    value=$(echo "$value" | sed 's/^"//' | sed 's/"$//')
    export "$key=$value"
done <<EOF
$(azd env get-values)
EOF

This code is redundant since azd will automatically makes the azd environment variables available to your script.

Question: Where are you looking to leverage the new variables you are creating?

A common scenario would be to have your preprovision script do something, set some azd environment variables then use these variables within your bicep module. This scenario could be supported by either configuring your main.parameters.json or using bicepparam files that can read environment variables.

[wbreza]

I was able to dive into this a little more. I have identified the root cause of the issue you are facing. In the up workflow we are initializing the provision provider too early which is cause azd to prompt for bicep parameters before the preprovision hook is run.

@weikanglim / @ellismg - We should really remove the provision provider from initializing in the up workflow and just let it kick in automatically during the provision step.

This line is the issue. https://github.com/Azure/azure-dev/blob/main/cli/azd/cmd/up.go#L113

[aaronpowell]

Some feedback regarding your linked script. I noticed you are running the following:

# Use the `get-values` azd command to retrieve environment variables from the `.env` file
while IFS='=' read -r key value; do
    value=$(echo "$value" | sed 's/^"//' | sed 's/"$//')
    export "$key=$value"
done <<EOF
$(azd env get-values)
EOF

This code is redundant since azd will automatically makes the azd environment variables available to your script.

I didn't realise it wasn't needed. It's possible that I just added it without testing, but I can remove it now.

Question: Where are you looking to leverage the new variables you are creating?

The scripts are being used to create an App Registration in Entra, since there's no Bicep support (or at least, there wasn't at the time and I haven't been able to test the beta that was released), and then register the app with it for authentication.

[aaronpowell]

I was able to dive into this a little more. I have identified the root cause of the issue you are facing. In the up workflow we are initializing the provision provider too early which is cause azd to prompt for bicep parameters before the preprovision hook is run.

@weikanglim / @ellismg - We should really remove the provision provider from initializing in the up workflow and just let it kick in automatically during the provision step.

This line is the issue. https://github.com/Azure/azure-dev/blob/main/cli/azd/cmd/up.go#L113

Yeah, we tend to use up out of sheer laziness (and that we've not created a GitHub Actions workflow yet 😜), so I always assumed that if we run the commands one-by-one then it wouldn't hit this, since a new run context is made each time.

[bronthulke]

Just on this

# Use the `get-values` azd command to retrieve environment variables from the `.env` file
while IFS='=' read -r key value; do
    value=$(echo "$value" | sed 's/^"//' | sed 's/"$//')
    export "$key=$value"
done <<EOF
$(azd env get-values)
EOF

This code is redundant since azd will automatically makes the azd environment variables available to your script.

I'm using PowerShell and I also have found I've had to use the output of azd env get-values to get to environment variables.

I have lines such as this:

$appServiceName = (azd env get-values --output json | ConvertFrom-Json).APP_SERVICE_NAME

How would you recommend we do this @wbreza ?

[wbreza]

@bronthulke azd environment variables are automatically made available when your hook script is running in the context of azd using standard environment variable conventions for your desired shell.

Assuming the following:

1. You have an azd environment already with some variable defined.

.azure/my-env/.env

MY_CUSTOM_VAR="some value"

With your hook script you can access this variable like the following:

With shell/bash script:

echo ${MY_CUSTOM_VAR}

With Powershell

Write-Output $env:MY_CUSTOM_VAR

To easily test your hook scripts we also have a convience command, azd hooks run <hook>. If you have a preprovision hooks wired up in your azure.yaml you could easily test it by calling azd hooks run preprovision. This will invoke any project or service level hooks that have this hook configured.

[bronthulke]

OK I'm not afraid to admit when I make a silly mistake... I kept trying to "check" the variable on a PS command line by simply doing

$env:MY_CUSTOM_VAR

I was forgetting to do an actual Write-Output for it. This works:

Write-Output $env:MY_CUSTOM_VAR

It does indeed work. and just tidies up that code a little.

That azd hooks run <hook> command is super helpful for testing, too, thanks!

[jongio]

Some feedback regarding your linked script. I noticed you are running the following:

# Use the `get-values` azd command to retrieve environment variables from the `.env` file
while IFS='=' read -r key value; do
    value=$(echo "$value" | sed 's/^"//' | sed 's/"$//')
    export "$key=$value"
done <<EOF
$(azd env get-values)
EOF

This code is redundant since azd will automatically makes the azd environment variables available to your script.

I didn't realise it wasn't needed. It's possible that I just added it without testing, but I can remove it now.

Question: Where are you looking to leverage the new variables you are creating?

The scripts are being used to create an App Registration in Entra, since there's no Bicep support (or at least, there wasn't at the time and I haven't been able to test the beta that was released), and then register the app with it for authentication.

We had this azd env load in early scripts before we had azd hooks run. Technically this load isn't needed if you are only running the script as a hook or with azd hooks run, but if you want to run the script directly (./script.sh) then you'll need this block to load the azd env.