Setting auto tune options in aws_elasticsearch_domain failing with 409 conflicting error

[anuj1366]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform AWS Provider Version

Terraform CLI: 1.2.0 TerraformAWS Provider Version : 4.19.0

Affected Resource(s)

• aws_elasticsearch_domain

Terraform Configuration Files

Please include all Terraform configurations required to reproduce the bug. Bug reports without a functional reproduction may be closed without investigation.

resource "aws_elasticsearch_domain" "opensearch" {
  domain_name           =   format("%s%s",var.domain_name,var.env)
  elasticsearch_version = var.opensearch_version

  access_policies = <<POLICY
  {
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": "es:*",
      "Principal":  {
        "AWS": "*"
      },
      "Effect": "Allow",
      "Resource": "arn:aws:es:${data.aws_region.current.name}:${data.aws_caller_identity.current.account_id}:domain/${var.domain_name}${var.env}/*"
    }
  ]
}
POLICY

  auto_tune_options {
    desired_state = "ENABLED"
    rollback_on_disable = "DEFAULT_ROLLBACK"
    maintenance_schedule {
      start_at = "2022-07-10T12:00:00Z"
      duration {
        value  = 2
        unit  = "HOURS"
      }
      cron_expression_for_recurrence = "cron(0 12 ? * 1 *)"
    }
  }

  cluster_config {
    instance_type = "r6g.large.elasticsearch"
    zone_awareness_config {
      availability_zone_count = 2
    }
    zone_awareness_enabled = true
    instance_count = var.data_node_instance_count
    
    dedicated_master_enabled = true
    dedicated_master_count  = 3
    dedicated_master_type = "m6g.large.elasticsearch"
    
    
  }

  ebs_options {
      ebs_enabled = true
      volume_size = var.ebs_storage_size
      volume_type = "gp2"
    }

  advanced_security_options {
      enabled = true
      internal_user_database_enabled = false
      master_user_options {
        master_user_arn  = var.iam_role_arn
      }
    }  

  domain_endpoint_options {
    enforce_https = true
    tls_security_policy = "Policy-Min-TLS-1-2-2019-07"
  }

  node_to_node_encryption {
      enabled = true
    }
  
  encrypt_at_rest {
      enabled = true
      kms_key_id = "alias/aws/es"
    }


  vpc_options {
    subnet_ids = [
      var.private_subnet_ids[0],
      var.private_subnet_ids[1],
    ]

    security_group_ids = [var.sg_id]
  }
  
  log_publishing_options {
    log_type                 = "SEARCH_SLOW_LOGS"
    enabled                  = true
    cloudwatch_log_group_arn = var.es_log_arn[0]
  }

  log_publishing_options {
    log_type                 = "INDEX_SLOW_LOGS"
    enabled                  = true
    cloudwatch_log_group_arn = var.es_log_arn[1]
  }

  log_publishing_options {
    log_type                 = "ES_APPLICATION_LOGS"
    enabled                  = true
    cloudwatch_log_group_arn = var.es_log_arn[2]
  }

  log_publishing_options {
    log_type                 = "AUDIT_LOGS"
    enabled                  = true
    cloudwatch_log_group_arn = var.es_log_arn[3]
  }

  tags = {
    "Name" = format("%s%s",var.domain_name,var.env)
    "owner" = "warriors"
    "environment" = var.env
    "application" = "tsi-central-logging"
    "tsi:data:classification" = "internal"
    "tsi:iac-type" = "TF"
    "tsi:iac-stack-name" = "security-monitoring"
  }
}

Debug Output

https://gist.github.com/anuj1366/611b3157e7d8d29833122f262747f7dd

Expected Behavior

Auto tune should be set along with maintenance window in OpenSearch domain without terraform script fail.

Actual Behavior

Error: error updating Elasticsearch Domain (arn:aws:es:eu-central-1:379494357693:domain/tsi-os-monitoring-dev) config: InvalidTypeException: To roll back, specify DesiredState as DISABLED.

Auto tune is setting along with maintenance window in OpenSearch domain as expected but terraform script is failing. If we remove the auto_tune_options block from terraform script, then opensearch domain is getting created successfully and scipt is also passing without any error.

Steps to Reproduce

1. terraform apply

Important Factoids

References

• #0000

[jsharper]

I just ran into this as well with AWS provider 3.75.2 and version OpenSearch_1.2

It appears that a workaround is to set rollback_on_disable = "NO_ROLLBACK".

As an aside, I'm only setting rollback_on_disable because of an error Error: Error modifying config for Elasticsearch domain: ValidationException: 1 validation error detected: Value '' at 'autoTuneOptions.rollbackOnDisable' failed to satisfy constraint: Member must satisfy enum value set: [DEFAULT_ROLLBACK, NO_ROLLBACK] when omitting it (despite docs indicating it is optional).

[nglcasanova]

I just ran into this as well with AWS provider 4.22.0 and version OpenSearch_1.2... I just tried @jsharper workaround (rollback_on_disable = "NO_ROLLBACK") and it worked.

[karlkatzke]

Also hit this with provider 4.22.0. Noting as @jsharper did that there's a conflict between documentation, which lists rollback_on_disable as optional, and actual state where rollback_on_disable is required.

Hitting a bug in OpenSearch/ElasticSearch is especially inconvenient because it takes 30 minutes to get an alteration through the API, only to find out that your terraform run failed due to a bug.