terraform-provider-digitalocean
terraform-provider-digitalocean copied to clipboard
digitalocean
`digitalocean_app` redeploys even though no `env` variables changed
Bug Report
Describe the bug
The digitalocean_app resource thinks that the env arguments are changing on every Terraform apply. It seems to only happen when the env are of type "SECRET" rather than "GENERAL".
Affected Resource(s)
- digitalocean_app
Expected Behaviour
It should not redeploy the app.
Actual Behaviour
It redeployed the app.
Steps to Reproduce
Once I have the below app deployed to Digital Ocean, if I run terraform apply all of the environment variables with the type "SECRET" show as being changed and it forces a rebuild.
Terraform Configuration Files
resource "digitalocean_app" "app" {
spec {
name = var.project_name
region = var.instance_region
domain {
name = var.api_domain_name
type = "PRIMARY"
zone = "[redacted].com"
}
env {
key = "DJANGO_SETTINGS_MODULE"
value = var.django_settings_module
type = "GENERAL"
}
env {
key = "PYTHONPATH"
value = "./[redacted]"
type = "GENERAL"
}
env {
key = "DATABASE_URL"
value = "$${db.DATABASE_URL}"
scope = "RUN_TIME"
type = "SECRET"
}
env {
key = "DOMAIN_NAME"
value = var.api_domain_name
type = "GENERAL"
}
env {
key = "BUCKET_NAME"
value = digitalocean_spaces_bucket.bucket.name
type = "GENERAL"
}
env {
key = "AWS_S3_ENDPOINT_URL"
value = "https://${var.bucket_region}.digitaloceanspaces.com"
type = "GENERAL"
}
env {
key = "AWS_ACCESS_KEY_ID"
value = var.spaces_access_id
type = "SECRET"
}
env {
key = "AWS_SECRET_ACCESS_KEY"
value = var.spaces_secret_key
type = "SECRET"
}
env {
key = "SENTRY_DSN"
value = var.sentry_dsn
type = "SECRET"
}
service {
name = "api"
environment_slug = "python"
instance_count = 1
instance_size_slug = var.api_instance_size
http_port = 8000
run_command = "gunicorn --worker-tmp-dir /dev/shm [redacted].wsgi"
github {
repo = "[redacted]"
branch = "main"
deploy_on_push = true
}
}
job {
name = "setup"
kind = "POST_DEPLOY"
environment_slug = "python"
instance_count = 1
instance_size_slug = "basic-xxs"
run_command = "python manage.py createcachetable; python manage.py migrate"
github {
repo = "[redacted]"
branch = "main"
deploy_on_push = true
}
}
database {
name = "db"
engine = "PG"
production = var.database_is_production
}
}
}
I'm running Terraform v1.1.4 and v2.17.0 of this library.
References
This seems to be the same issue: https://github.com/digitalocean/terraform-provider-digitalocean/issues/514
The two most recent commenters after the issue closed seem to be experiencing a similar issue to me.
Any new information? We are facing this issue and new it solved, please. Thank you
Thank you all for bringing this issue to our attention.
@jameyg can you confirm that the env vars with type = "SECRET" in your config have respective encrypted values being passed in (typically through a tfvars file).
The fix mentioned in the #514 was to Suppress diff when env type is GENERAL. However, if the values for any SECRET vars are not the same encrypted values returned by the API, it will result in a diff.
I will attempt to reproduce this issue to investigate further.
@scotchneat are you able to reproduce the problem and has an idea when a fix could be releases ?
@driedel1987 yes.
When using the env vars with type = "SECRET" as I mentioned in my previous comment, app platform will encrypt and store the secret, then returns the encrypted value in the response. Terraform then stores the encrypted value in state.
The problem is, if the same plain-text value sent in the initial apply is sent on any subsequent apply, the values won't match the encrypted value stored in the state so it creates a diff. This is expected behavior and not a bug in the provider.
I realize this scenario isn't properly covered in the documentation. I'll see if I can elaborate on this in the documentation and probably an example config
@scotchneat thanks for your answer. But I have the same behavior if i only use “general” as type in my app (https://github.com/pulumi/pulumi-digitalocean/issues/308).
I am using the following code to roll out my app/worker in digital ocean:
func RegisterApp(UserID, EncryptionPassPhrase, AwsAccessKeyID, AwsGENERALAccessKey, AwsRegion, SlackWebhookDomain, SlackTeam, SlackService, SlackToken, ShedulerImageTag, CloudFlareDomain, AppAccessKeyID, AppBotToken, AppClientID, AppClientGENERAL, AppSpaceEndpoint, AppGENERALAccessKey, AppServerIP, AppPort, CustomerDBHost, CustomerDBPort, AppCA, AppImageTag, DashboardCorsDomain string, di *models.DatabaseUserInformation, ai *models.AppUserInformation) (pulumi.RunFunc, error) {
return func(ctx *pulumi.Context) error {
App, err := digitalocean.NewApp(ctx, UserID, &digitalocean.AppArgs{
Spec: &digitalocean.AppSpecArgs{
Name: pulumi.String(UserID),
Region: pulumi.String("fra"),
DomainNames: &digitalocean.AppSpecDomainNameArray{digitalocean.AppSpecDomainNameArgs{Name: pulumi.String(ai.Name + CloudFlareDomain)}},
Workers: &digitalocean.AppSpecWorkerArray{&digitalocean.AppSpecWorkerArgs{
Name: pulumi.String("golang-worker"),
Image: &digitalocean.AppSpecWorkerImageArgs{RegistryType: pulumi.String("DOCR"), Repository: pulumi.String("sheduled-job"), Tag: pulumi.String(ShedulerImageTag)},
InstanceCount: pulumi.Int(1),
InstanceSizeSlug: pulumi.String("basic-xxs"),
Alerts: &digitalocean.AppSpecWorkerAlertArray{&digitalocean.AppSpecWorkerAlertArgs{Rule: pulumi.String("CPU_UTILIZATION"), Operator: pulumi.String("GREATER_THAN"), Value: pulumi.Float64(85), Window: pulumi.String("FIVE_MINUTES")},
&digitalocean.AppSpecWorkerAlertArgs{Rule: pulumi.String("MEM_UTILIZATION"), Operator: pulumi.String("GREATER_THAN"), Value: pulumi.Float64(85), Window: pulumi.String("FIVE_MINUTES")},
&digitalocean.AppSpecWorkerAlertArgs{Rule: pulumi.String("RESTART_COUNT"), Operator: pulumi.String("GREATER_THAN"), Value: pulumi.Float64(5), Window: pulumi.String("FIVE_MINUTES")}},
Envs: &digitalocean.AppSpecWorkerEnvArray{
&digitalocean.AppSpecWorkerEnvArgs{
Key: pulumi.String("DB_HOST"),
Scope: pulumi.String("RUN_AND_BUILD_TIME"),
Type: pulumi.String("GENERAL"),
Value: pulumi.String(CustomerDBHost),
},
&digitalocean.AppSpecWorkerEnvArgs{
Key: pulumi.String("DB_PORT"),
Scope: pulumi.String("RUN_AND_BUILD_TIME"),
Type: pulumi.String("GENERAL"),
Value: pulumi.String(CustomerDBPort),
},
...............
It creates the app/worker successfully in digital ocean without using any type of secret.
But if I try an update directly after the creation of the app, Pulumi recognizes changes which are not present. I am using the same function for updating as I am using for creation (see above).
The Preview shows the following output in terminal for example: It seems that Pulumi wants to change the order or something like that. If I execute the preview, it runs in a timeout.
~ spec: {
~ services: [
~ [0]: {
~ envs : [
~ [0]: {
~ key : "CUSTOMERID" => "CORS_DOMAIN"
~ scope: "RUN_AND_BUILD_TIME" => "RUN_AND_BUILD_TIME"
~ value: [secret] => "http://localhost:3000"
}
~ [1]: {
~ key : "BOT_TOKEN" => "CLIENT_ID"
~ scope: "RUN_AND_BUILD_TIME" => "RUN_AND_BUILD_TIME"
~ value: [secret] => "Value"
}
~ [2]: {
~ key : "DB_HOST" => "AWS_SES_ACCESS_KEY_ID"
~ scope: "RUN_AND_BUILD_TIME" => "RUN_AND_BUILD_TIME"
~ value: [secret] => "Value"
}
~ [3]: {
~ key : "DB_PORT" => "AWS_SES_GENERAL_ACCESS_KEY"
~ scope: "RUN_AND_BUILD_TIME" => "RUN_AND_BUILD_TIME"
~ value: [secret] => "Value"
}
~ [4]: {
~ key : "EncryptionPassPhrase" => "AWS_SES_REGION"
~ scope: "RUN_AND_BUILD_TIME" => "RUN_AND_BUILD_TIME"
~ value: [secret] => "eu-central-1"
Thanks for the details @driedel1987.
I realize you're sharing only a snippet of the code and the diff output, but what stands out in the planned change is that the env vars listed in your code are supposed to be changed to other values (e.g. "DB_HOST" => "AWS_SES_ACCESS_KEY_ID"). This makes me think the pulumi library has some dependency on the order of those values. From my testing, plain terraform doesn't diff when env vars order changes.
However, since that's beyond the function or control of this provider, I'd like to avoid confusing the experience from the initial request in this issue with the behavior/interaction between pulumi and the DO Terraform provider. In the pulumi issue you referenced, @stack72 points to this issue being related to the behavior experienced in the pulumi library. @stack72 could you please share more details about the interaction? If it is beyond the experience described by the initial reporter of this issue, I think it makes more sense that you create a new issue so the topic can be specific to the pulumi > digitalocean provider integration.
are there any workarounds for the SECRET env vars always triggering changes? every time I apply my terraform it causes the app being deployed even though nothing changed
@scotchneat Sorry I didn't see the notifications for this come through.
I ended up changing everything to "general" rather than "secret" which almost fixed my issue but not completely. I have three environments (dev/staging/prod) powered by the same module and for some reason the staging environment always retriggers a change and deployment stating that something in the env block has changed. However, if that were the case then both the other two environments should also be triggering a change and deployment.
I'm not sure what files or information to attach but I'm using Terraform without any abstractions beyond it running in a Docker container locally.
Any updates or new workarounds on this? I am still facing this issue.
We've pushed up an example that offers a workaround in #1030
We are aware this workaround adds a few more manual steps, but it should mitigate the issue enough until we find a better solution. Please feel free to offer feedback in that PR.
