hashicorp
Manual change in configurations will clobber/corrupt terraform state.
Describe the bug
Terraform state files are not saved locally, hence if some configuration is changed/deleted manually on a10 thunder, they are never applied using consul-terraform-sync.
Versions
consul-nia 44fc2bb (44fc2bb)
Compatible with Terraform ~>0.13.0
Consul Version
Consul 1.8.4
Terraform Version
Terraform v0.13.4
Configuration File(s)
No changes on terraform apply if some configuration is changes on thunder CLI.
log_level = "INFO"
consul {
address = "<Consul-server-IP>:8500"
}
driver "terraform" {
log = true
required_providers {
thunder = {
source = "a10networks/thunder"
#version = "0.4.14"
}
}
}
provider "thunder" {
address = "a10-thunder-mgmt-ip"
alias = "thunder1"
username = "<thunder-username>"
password = "<thunder-password>"
}
task {
name = "automate-services1"
description = "automate services for website"
source = "a10networks/service-group-sync-nia/thunder"
#version = "0.1.3"
providers = ["thunder.thunder1"]
services = ["web","api"]
}
Terraform Configuration Files Generated by Consul-Terraform-Sync
Reminder to redact any sensitive information that may be present in the files
Click to toggle contents of main.tf
# This file is generated by Consul NIA.
#
# The HCL blocks, arguments, variables, and values are derived from the
# operator configuration for Consul NIA. Any manual changes to this file
# may not be preserved and could be clobbered by a subsequent update.
terraform {
required_version = "~>0.13.0"
required_providers {
thunder = {
source = "a10networks/thunder"
}
}
backend "consul" {
address = "<Consul-server-IP>:8500"
gzip = true
path = "consul-nia/terraform"
}
}
provider "thunder" {
address = var.thunder.address
password = var.thunder.password
username = var.thunder.username
}
# automate services for website
module "automate-services1" {
source = "a10networks/service-group-sync-nia/thunder"
services = var.services
}
Click to toggle contents of terraform.tfvars
# This file is generated by Consul NIA.
#
# The HCL blocks, arguments, variables, and values are derived from the
# operator configuration for Consul NIA. Any manual changes to this file
# may not be preserved and could be clobbered by a subsequent update.
thunder = {
address = "10.65.22.161"
alias = "thunder1"
password = "<thunder-password>"
username = "<thunder-username>"
}
services = {
"api2" : {
id = "api2"
name = "api"
address = "172.31.75.182"
port = 8090
meta = {}
tags = ["api", "dc1", "test"]
namespace = null
status = "passing"
node = "i-03c86bc24c675126b"
node_id = "34c7c895-fbb9-1a86-5f47-75283a349d81"
node_address = "172.31.75.182"
node_datacenter = "us-east-1"
node_tagged_addresses = {
lan = "172.31.75.182"
lan_ipv4 = "172.31.75.182"
wan = "172.31.75.182"
wan_ipv4 = "172.31.75.182"
}
node_meta = {
consul-network-segment = ""
}
},
"api0" : {
id = "api0"
name = "api"
address = "172.31.7.19"
port = 8090
meta = {}
tags = ["api", "dc1", "test"]
namespace = null
status = "passing"
node = "i-051eb4d7c2ab0e661"
node_id = "a78f2ff3-91aa-90aa-6f39-4726522e872e"
node_address = "172.31.7.19"
node_datacenter = "us-east-1"
node_tagged_addresses = {
lan = "172.31.7.19"
lan_ipv4 = "172.31.7.19"
wan = "172.31.7.19"
wan_ipv4 = "172.31.7.19"
}
node_meta = {
consul-network-segment = ""
}
},
"api1" : {
id = "api1"
name = "api"
address = "172.31.52.231"
port = 8090
meta = {}
tags = ["api", "dc1", "test"]
namespace = null
status = "passing"
node = "i-054002ad10be452a7"
node_id = "8f5e3fad-b5f5-bb89-409e-e52dcfb97314"
node_address = "172.31.52.231"
node_datacenter = "us-east-1"
node_tagged_addresses = {
lan = "172.31.52.231"
lan_ipv4 = "172.31.52.231"
wan = "172.31.52.231"
wan_ipv4 = "172.31.52.231"
}
node_meta = {
consul-network-segment = ""
}
},
"web1" : {
id = "web1"
name = "web"
address = "172.31.52.231"
port = 80
meta = {}
tags = ["dc1", "nginx", "test", "web"]
namespace = null
status = "passing"
node = "i-054002ad10be452a7"
node_id = "8f5e3fad-b5f5-bb89-409e-e52dcfb97314"
node_address = "172.31.52.231"
node_datacenter = "us-east-1"
node_tagged_addresses = {
lan = "172.31.52.231"
lan_ipv4 = "172.31.52.231"
wan = "172.31.52.231"
wan_ipv4 = "172.31.52.231"
}
node_meta = {
consul-network-segment = ""
}
},
"web0" : {
id = "web0"
name = "web"
address = "172.31.10.35"
port = 80
meta = {}
tags = ["dc1", "nginx", "test", "web"]
namespace = null
status = "passing"
node = "i-0bff749211d19957b"
node_id = "23722d1e-2c3b-ed68-8a7f-d14e2e2c4964"
node_address = "172.31.10.35"
node_datacenter = "us-east-1"
node_tagged_addresses = {
lan = "172.31.10.35"
lan_ipv4 = "172.31.10.35"
wan = "172.31.10.35"
wan_ipv4 = "172.31.10.35"
}
node_meta = {
consul-network-segment = ""
}
}
}
Terraform Module
https://github.com/a10networks/terraform-thunder-service-group-sync-nia
Task Variable Files
If passing in task variable file(s), share relevant parts of your variable file(s) here.
Expected Behavior
All the service information fetched from the consul bootstrap server should be applied properly to thunder.
Actual Behavior
If consul-terraform-sync is stopped in between and restarted again, then no new config change is applied to the thunder.
Steps to Reproduce
1.) Once consul bootstrap server is configured, start consul-terraform-sync process 2.) Services are fetched and applied properly 3.) Stop the consul-terraform-sync using Ctrl+c 4.) Remove configuration from a10's thunder virtual-machine 5.) start the process with same command as in step 1, since state files are saved centrally somewhere therefore no change/services are updated on a10 thunder virtual-machine 6.) From this point onwards only updated services are applied.
Additional Context
There should be some way to delete the previously generated state files. Only way we have now is to give different task name and retry as this might generate separated set of tfstate files on centralized server where tfstate files are saved.
Hi @asharma-a10! Thanks for sending this issue report.
I have a few clarification questions to better understand how to address the issue. What kind of manual changes are being made on A10 thunder and do they relate to the service groups and member lists created by the a10networks/terraform-thunder-service-group-sync-nia module? My concern is that it might be out of scope for CTS to be able to resolve the out of band changes made directly to the infrastructure. This is a common Terraform caveat that changes made outside of Terraform to the resources managed by Terraform introduces complicated states and drift that the provider may or may not be able to detect or resolve.
You did bring up a great point about management of state files. There currently is not a friendly way to delete state or restart a task from fresh (to reset the state and pick up those manual changes). Giving it a different task name is a work-around but understandably not ideal as it'll litter the state store and effectively reserve that task name from being reused.
To mitigate the Terraform caveat, I'm interested in hearing what kind of behavior you would expect with deleting a state file: would you imagine the tracked resources being destroyed or left-as-is, or another behavior?
