elastic
Improve documentation around configuring deletion_protection = false
Clarify the intended workflow required to delete/replace indices managed with deletion_protection=true.
Original issue
Describe the bug
If you create an index initially with deletion_protection = true, you are unable to delete it after changing to deletion_protection = false.
It keeps returning:
Error: cannot destroy index without setting deletion_protection=false and running terraform apply
Same behaviour if you initially create the index with deletion_protection = false; after you turn it to true, you are still able to delete the index without any error, which should be prevented.
To Reproduce Steps to reproduce the behavior:
- TF configuration used:
terraform {
required_version = ">= 1.0.0"
required_providers {
elasticstack = {
source = "elastic/elasticstack"
version = "~>0.9"
}
}
}
provider "elasticstack" {
elasticsearch {
endpoints = ["http://elastic01.internal:9200"]
username = var.admin_username
password = var.admin_password
}
}
resource "elasticstack_elasticsearch_index" "index-0001" {
name = "index-0001"
deletion_protection = true
mappings = jsonencode({
properties = {
field1 = { type = "date" }
}
})
number_of_shards = 1
number_of_replicas = 0
}
- TF operations to execute to get the error:
terraform apply- Change any value, like mapping field type, to force replacement, and also set
deletion_protection = false terraform apply
- See the error in the output:
elasticstack_elasticsearch_index.index-0001: Destroying... [id=SAv5uMFYSA-prywrU6dE_g/index-0001]
│ Error: cannot destroy index without setting deletion_protection=false and running `terraform apply`
Expected behavior
It should be able to destroy the index when deletion_protection changed to false.
Versions:
- OS: MacOS
- Terraform Version: 1.9.3
- Provider Version: 0.11.4
- Elasticsearch Version: 8.14.3
This behaviour was intentional by the original author of this part of the code. It forces a 2 phased application for any action resulting in index deletion and IIRC mimics the behaviour on the GCP provider (and potentially others with similar attributes).
The intended workflow is:
terraform applywithdeletion_protection=true- Reset
deletion_protectionbyterraform applywithdeletion_protection=falseand not changes forcing replacement - Delete the index or apply changes requiring replacement (like mapping field type).
I've ran into this deadlock, most likely because I've used a resource lifecycle, here's my resource definition:
resource "elasticstack_elasticsearch_index" "index_bootstraped" {
for_each = var.index
name = "${each.value.name}-000001"
alias {
name = each.value.name
is_write_index = true
}
lifecycle {
ignore_changes = [name]
}
}
Then after noticing that I needed to get rid of that resource, I've only added deletion_protection = false to the resource, but the plan/apply presents me that replacement needs to happen because of the deletion protection:
# module.es_index.elasticstack_elasticsearch_index.index_bootstraped["logs-ecs"] is tainted, so must be replaced
-/+ resource "elasticstack_elasticsearch_index" "index_bootstraped" {
~ deletion_protection = true -> false
~ id = "HwB2f7HKRcGq8HBkGSFraQ/logs-ecs-000001" -> (known after apply)
~ mappings = jsonencode(
~ {
- dynamic = "strict"
- dynamic_templates = [
- {
- ecs_timestamp = {
- mapping = {
- ignore_malformed = false
- type = "date"
}
- match = "@timestamp"
}
},
- {
- ecs_message_match_only_text = {
- mapping = {
- type = "match_only_text"
}
- path_match = [
- "message",
- "*.message",
]
- unmatch_mapping_type = "object"
}
},
- {
- ecs_non_indexed_keyword = {
- mapping = {
- doc_values = false
- index = false
- type = "keyword"
}
- path_match = "event.original"
}
},
- {
- ecs_non_indexed_long = {
- mapping = {
- doc_values = false
- index = false
- type = "long"
}
- path_match = "*.x509.public_key_exponent"
}
},
- {
- ecs_ip = {
- mapping = {
- type = "ip"
}
- match_mapping_type = "string"
- path_match = [
- "ip",
- "*.ip",
- "*_ip",
]
}
},
- {
- ecs_wildcard = {
- mapping = {
- type = "wildcard"
}
- path_match = [
- "*.io.text",
- "*.message_id",
- "*registry.data.strings",
- "*url.path",
]
- unmatch_mapping_type = "object"
}
},
- {
- ecs_path_match_wildcard_and_match_only_text = {
- mapping = {
- fields = {
- text = {
- type = "match_only_text"
}
}
- type = "wildcard"
}
- path_match = [
- "*.body.content",
- "*url.full",
- "*url.original",
]
- unmatch_mapping_type = "object"
}
},
- {
- ecs_match_wildcard_and_match_only_text = {
- mapping = {
- fields = {
- text = {
- type = "match_only_text"
}
}
- type = "wildcard"
}
- match = [
- "*command_line",
- "*stack_trace",
]
- unmatch_mapping_type = "object"
}
},
- {
- ecs_path_match_keyword_and_match_only_text = {
- mapping = {
- fields = {
- text = {
- type = "match_only_text"
}
}
- type = "keyword"
}
- path_match = [
- "*.title",
- "*.executable",
- "*.name",
- "*.working_directory",
- "*.full_name",
- "*file.path",
- "*file.target_path",
- "*os.full",
- "email.subject",
- "vulnerability.description",
- "user_agent.original",
]
- unmatch_mapping_type = "object"
}
},
- {
- ecs_date = {
- mapping = {
- type = "date"
}
- path_match = [
- "*.timestamp",
- "*_timestamp",
- "*.not_after",
- "*.not_before",
- "*.accessed",
- "created",
- "*.created",
- "*.installed",
- "*.creation_date",
- "*.ctime",
- "*.mtime",
- "ingested",
- "*.ingested",
- "*.start",
- "*.end",
- "*.indicator.first_seen",
- "*.indicator.last_seen",
- "*.indicator.modified_at",
- "*threat.enrichments.matched.occurred",
]
- unmatch_mapping_type = "object"
}
},
- {
- ecs_path_match_float = {
- mapping = {
- type = "float"
}
- path_match = [
- "*.score.*",
- "*_score*",
]
- path_unmatch = "*.version"
- unmatch_mapping_type = "object"
}
},
- {
- ecs_usage_double_scaled_float = {
- mapping = {
- scaling_factor = 1000
- type = "scaled_float"
}
- match_mapping_type = [
- "double",
- "long",
- "string",
]
- path_match = "*.usage"
}
},
- {
- ecs_geo_point = {
- mapping = {
- type = "geo_point"
}
- path_match = "*.geo.location"
}
},
- {
- ecs_flattened = {
- mapping = {
- type = "flattened"
}
- match_mapping_type = "object"
- path_match = [
- "*structured_data",
- "*exports",
- "*imports",
]
}
},
- {
- all_strings_to_keywords = {
- mapping = {
- ignore_above = 1024
- type = "keyword"
}
- match_mapping_type = "string"
}
},
]
}
)
name = "logs-ecs-000001"
~ settings_raw = jsonencode(
{
- "index.creation_date" = "1732288963831"
- "index.lifecycle.name" = "logs"
- "index.lifecycle.rollover_alias" = "logs-ecs"
- "index.number_of_replicas" = "1"
- "index.number_of_shards" = "2"
- "index.provided_name" = "logs-ecs-000001"
- "index.routing.allocation.include._tier_preference" = "data_content"
- "index.uuid" = "sUJh4ZTCQ6qo859pSD9GRQ"
- "index.version.created" = "8512000"
}
) -> (known after apply)
# (4 unchanged attributes hidden)
# (1 unchanged block hidden)
}
Plan: 1 to add, 0 to change, 1 to destroy.
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
module.es_index.elasticstack_elasticsearch_index.index_bootstraped["logs-ecs"]: Destroying... [id=HwB2f7HKRcGq8HBkGSFraQ/logs-ecs-000001]
╷
│ Error: cannot destroy index without setting deletion_protection=false and running `terraform apply`
│
│ cannot destroy index without setting deletion_protection=false and running `terraform apply`
Now I'll probably have to remove it from the state file
And for the record, I've added the resource lifecycle in an attempt to deal with the fact that we bootstrap the indexes only once, and eventually the 000001 will go away with time
but the plan/apply presents me that replacement needs to happen because of the deletion protection:
# module.es_index.elasticstack_elasticsearch_index.index_bootstraped["logs-ecs"] is tainted, so must be replaced
The index is being recreated because it's tainted in state, not because deletion protection has been changed.