Unclear how to add and manage secrets

[dluc]

Is there an existing issue for this?

• [x] I have searched the existing issues

Describe the bug

After an initial deployment I added a couple of secrets to a project, following this doc https://learn.microsoft.com/en-us/dotnet/aspire/fundamentals/external-parameters

• added accesskey1 and accesskey2 Parameters in AppHost's appsettings.json
• set values using dotnet user-secrets set "Parameters:accesskey1" "..." and dotnet user-secrets set "Parameters:accesskey2" "..."
• run azd deploy

The deployment kept failing with:

ERROR: failed deploying service 'myResource': failed executing template file: template: manifest template:25:19: executing "manifest template" at <securedParameter "accesskey1">: error calling securedParameter: parameter accesskey1 not found

After trying different approaches, I ran azd provision and it prompted for the two secrets, updating .azure/{envname}/config.json, adding the two new entries:

{
  "infra": {
    "parameters": {
      "accesskey1": "vault://11073122-0831-4cba-9ed8-17ec4360fde8/aaa665e4-ac99-46c7-acd8-a36d44e7bc32",
      "accesskey2": "vault://11073122-0831-4cba-9ed8-17ec4360fde8/26b3d840-c3ff-491a-bf38-cf893bcc6714",
      "qdrantstorage_Key": "vault://11073122-0831-4cba-9ed8-17ec4360fde8/cd946f29-c6b9-4003-9701-f164de0926eb"
    }
  },
  "vault": "11073122-0831-4cba-9ed8-17ec4360fde8"
}

My deployment doesn't include any AKV. The vault is stored locally on my workstation, under ~/.azd/vaults/11073122-0831-4cba-9ed8-17ec4360fde8.json, where the secrets are saved in base64 format:

{
  "26b3d840-c3ff-491a-bf38-cf893bcc6714": "cGFzc3dvcmQ=",
  "aaa665e4-ac99-46c7-acd8-a36d44e7bc32": "cGFzc3dvcmQ=",
  "cd946f29-c6b9-4003-9701-f164de0926eb": "Zm9vYmFy"
}

azd deploy is reading secrets from this file and ignoring those set with dotnet user-secrets, which seems to deviate from the documentation.

Additionally, azd env set-secret "accesskey1" prompts for a KeyVault from Azure, rather than updating this local file. I haven't found how to update these secrets with azd so I'm editing the vault file manually encoding/decoding these base64 values.

Forcing values

FYI, I tried this but the values are ignored. The secrets in Azure portal are the values coming from AZD vault:

var key1 = builder.AddParameter("accesskey1", secret: true, value: "accesskey 1");
var key2 = builder.AddParameter("accesskey2", secret: true, value: "accesskey 2");

myResource
    .WithEnvironment("App__Authorization__AccessKey1", key1)
    .WithEnvironment("App__Authorization__AccessKey2", key2)

.NET Version info

azd version 1.14.0 (commit c928795c47f27d1e997c217147dc649054ac05c8) dotnet 9.0.202 Aspire.AppHost.Sdk 9.1.0

[davidfowl]

Yea this part of aspire + azd is really messy. Anything defined in the app host's configuration is used for development not deployment.

[davidfowl]

cc @vhvb1989 This is something we should spend some time on.

[davidfowl]

I'm not really sure that secrets should ever be pushed from your local environment to the deployed environment by default. That should be an explicit operation with potentially different values from whatever was in user secrets. I can see the argument being that maybe azd provision/deploy is about moving from local dev to remote dev so it can use the same values.

[ekomsctr]

Yea this part of aspire + azd is really messy. Anything defined in the app host's configuration is used for development not deployment.

This. I'm using the Publish feature from Visual Studio, and when i create a new Environment, the IDE prompt me for the Parameters value, which are stored then in the config.json file in the .azure/ folder.

After that, when i deploy the aspire solution to ACA (both from right click > Publish and running azd up), the only Parameters used are the ones from the AppHost appsettings.Development.json, not even the appsettings.json are being considered.

Can we shed some light on this? It gets tiring replacing manually some configured secrets every time.

[davidfowl]

After that, when i deploy the aspire solution to ACA (both from right click > Publish and running azd up), the only Parameters used are the ones from the AppHost appsettings.Development.json, not even the appsettings.json are being considered.

Can you clarify what you mean by this? azd should be using config.json for parameter values, are you seeing something different?

[ekomsctr]

I made some test, here's what i found:

• Launch profile (as to be expected) define the DOTNET_ENVIRONMENT, which determines the appsettings.json the Parameters will be fetched from during deployment (or aspire-manifest generation)
• The "Right Click > Publish..." deployment seems to be using only the Parameters supplied in the corresponding appsettings..json
• The parameters supplied in the config.json values are not taken in consideration

Let me know how i can supply you better insights to understand the issue!

[davidfowl]

@bradygaster @vhvb1989 do the parameters in visual studio come from appsettings.json somehow? I don't see how that can happen unless VS is explicitly reading them from there and giving those values to azd.

[vhvb1989]

cc @vhvb1989 This is something we should spend some time on.

@davidfowl if you can express this in the manifest, AZD can take the user-secrets from dotnet during deployment. For example:

"param-to-user-secret": {
      "type": "parameter.v0",
      "value": "{param-to-user-secret.inputs.value}",
      "inputs": {
        "value": {
          "type": "string",
          "secret": true,
          "default":{
            "value": "{dotnet.user-secrets.secret-name}"
          }
        }
      }
    }

This would be telling anyone deploying that the value for the param-to-user-secret can be defined/overridden BUT, by default, it is mapped to dotnet user secret: secret-name. AZD would respect any override for the parameter or, when not explicitly set, it would run dotnet user-secrets list to fetch the value for the parameter.

[JohannTIM]

@vhvb1989 is it already possible to do these changes and force manifest to look into user-secrets we are struggling with managing the secrets from one place in the pipelines and are following this and #3597.

We would like to specifiy secrets in github to control the parameters there, and have not found a good method of doing it.

[vhvb1989]

@vhvb1989 is it already possible to do these changes and force manifest to look into user-secrets we are struggling with managing the secrets from one place in the pipelines and are following this and #3597.

We would like to specifiy secrets in github to control the parameters there, and have not found a good method of doing it.

@JohannTIM making manifest to describe a parameter linked to a user-secret is not currently possible. We would need to see if @davidfowl finds it doable and make it happen.

For controlling parameters with github secrets/variables, I have this PR open: https://github.com/Azure/azure-dev/pull/5143 It will give you what you are describing. Each parameter will be linked to either a secret or a variable in you CI

[JohannTIM]

Damn it - this is what I wanted for some of our secret parameters. Thanks for the update and crossing fingers 🙂

[vhvb1989]

This was handled similar than https://github.com/dotnet/aspire/issues/7680#issuecomment-2879303852

@davidfowl I think you can close this now?