A standard or better way to populate local environments with azd env variables

[pamelafox]

We currently have many templates that need access to azd environment variables to be able to run either hooks, scripts, or local dev server.

There are two ways that templates often do that:

Write the full azd env into a .env file, and then load it with a language package like python-dotenv:

azd env get-values > .env

Use shell commands to write the env variables into the environment, and call programs from the shell script:

Write-Host "Loading azd .env file from current environment"
foreach ($line in (& azd env get-values)) {
    if ($line -match "([^=]+)=(.*)") {
        $key = $matches[1]
        $value = $matches[2] -replace '^"|"$'
	    [Environment]::SetEnvironmentVariable($key, $value)
    }
}

Why those are bad

Both of these approaches are problematic as they can leak the azd environment variables into the global environment.

For example, the " > .env" approach leaks into the global environment when you're using the Python extension, as that extension (as a default behavior) automatically copies .env variables into the global environment. It took me months to figure out why my global env was getting tainted constantly.

The Powershell code above can also leak into the Windows shell, depending on how the rest of the script issues commands.

It is very bad when the full azd env variables leak into a global environment, since they include AZURE_ENV_NAME, AZURE_LOCATION, AZURE_SUBSCRIPTION_ID. If you then try to switch environments, you will find azd constantly trying to deploy with the values of the old environment. It's very confusing and caused me days of work over the last year trying to figure out what was happening.

Better approaches

I am now taking one of two approaches:

1. Using a script to auto-write only the necessary variables, and making sure those variables aren't also inputs in main.parameters.json: https://github.com/Azure-Samples/azure-openai-keyless-python/pull/7/files#diff-129e0db6b0e28f105813de4b3029d708f8012191253104aaadc5086e69a51aa3

That's not super robust, since it has the constraint that you can't also have those variables as inputs, but it can work for some simple samples.

2. Using a Python script to dynamically load in the current azd environment, using python-dotenv, so that it only ever is used inside that Python program..

https://github.com/Azure-Samples/azure-search-openai-demo/pull/1986/files#diff-6099ee740b8b4a7f97ac1e1dfff11776df721ed84c635e510c9aba8f922ca612

That is my current preferred approach, though it has the drawback of feeling a little overly complex for samples that are designed as teaching samples.

3. We provide vscode tasks that use the azd-provided dotenv as well, but that only works if you're running from VS Code, and we need to provide non-VS Code scripts as well.

EVEN better approaches??

These are related issues and PRs around this issue:

https://github.com/Azure/azure-dev/pull/4078

https://github.com/Azure/azure-dev/pull/4131

https://github.com/Azure/azure-dev/issues/1163

https://github.com/Azure/azure-dev/issues/4067