Datasource aws_opensearch_domain fails when advanced_security_options is enabled

[fmartinou]

Terraform CLI and Terraform AWS Provider Version

Terraform v1.2.8
on linux_amd64
+ provider registry.terraform.io/hashicorp/aws v4.29.0

Affected Resource(s)

• Datasource aws_opensearch_domain

Terraform Configuration Files

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "4.29.0"
    }
  }
}

provider "aws" {
  region = "eu-west-3"
}

resource "aws_opensearch_domain" "bug" {
  domain_name = "bug"

  cluster_config {
    instance_type            = "t3.small.search"
    instance_count           = 1
    dedicated_master_enabled = false
  }
  ebs_options {
    ebs_enabled = true
    volume_size = 10
    volume_type = "gp2"
  }
  node_to_node_encryption {
    enabled = true
  }
  encrypt_at_rest {
    enabled = true
  }
  domain_endpoint_options {
    enforce_https       = true
    tls_security_policy = "Policy-Min-TLS-1-2-2019-07"
  }
  advanced_security_options {
    enabled                        = true
    internal_user_database_enabled = true
    master_user_options {
      master_user_name     = "elastic"
      master_user_password = "Eel@st1c"
    }
  }
}

data "aws_opensearch_domain" "my_domain" {
  domain_name = aws_opensearch_domain.bug.domain_name
}

Debug Output

...
aws_opensearch_domain.bug: Creation complete after 16m16s [id=arn:aws:es:eu-west-3:560417158718:domain/bug]
data.aws_opensearch_domain.my_domain: Reading...
╷
│ Error: error setting advanced_security_options: Invalid address to set: []string{"advanced_security_options", "0", "anonymous_auth_enabled"}
│ 
│   with data.aws_opensearch_domain.my_domain,
│   on main.tf line 47, in data "aws_opensearch_domain" "my_domain":
│   47: data "aws_opensearch_domain" "my_domain" {
│ 
╵

See full log in attachment

References

This configuration is perfectly working with provider version 4.28.0

Maybe related to this PR?

[dpattmann]

Hi @fmartinou, you are right. The PR #26503 is related to this issue. The function flattenAdvancedSecurityOptions now tries to set anonymous_auth_enabled to a boolean value but the field is missing in the data resource model. I'm going to create a PR to fix this.

[fmartinou]

Any news regarding this fix? 😢

[xfournet]

This issue is a regression because it's preventing to reference an opensearch domain that has been created by terraform in another configuration. That is our case, and it prevent us to update, we are stuck to 4.28.0 since 2 months.

As this bug is a regression and the PR available (#26746 which is quite small), would be possible to prioritize it? It's really really annoying not be able to update, we are missing functionnalities that are in newer realeases

[tgeo-cambrian]

any news on this? creates an issue in our setup too.

[axi43]

We just upgraded and got impacted. Thanks so much for working on fixing this!

[Wyifei]

Any update for this?

[kevinashaw]

Updates? I can confirm that 4.28 works fine. But 4.48 fails with the following error:

│ Error: error setting advanced_security_options: Invalid address to set: []string{"advanced_security_options", "0", "anonymous_auth_enabled"}

Other than downgrading, I'm not sure how else to proceed.

[neovatar]

We encountered the same problem and are stuck with an old provider version - a merge and release of the PR would really be appreciated!

[barthel]

Any updates?

[spliskamatyshak-w3]

@YakDriver, it seems this has fallen on deaf ears. This is affecting more and more each day, now including my company.

[leonardolacerdaatlantico]

Any updates? I had same problem here. In really this is affecting more and more each day. Now, I'm blocked too in a old version.

[kaiohenricunha]

Any updates? It is a real drawback, but there is a PR ready to be merged and fix the issue. How come this issue has persisted for so long?

[mmcmaster-seek]

I also encountered this issue today on 4.54.0. I got past it by downgrading to 4.28.0.

[spliskamatyshak-w3]

Downgrading isn't an option for my situation. What I needed to look up, I've just stored programmatically in parameter store. It would be nice if the data call just worked properly with the latest provider, though.

[xfournet]

We are about to hit the wall in few days... We are stuck to v4.28.0 due to this regression. Since we are also deploying WAF rules with Terraform we will be blocked at the end of the month since at that time AWS will require WAF rules to specify Oversize Handling which is only possible starting from v4.52.0 (see #25545)

It's very questioning to me why Terraform, unlike the vast majority of open source project, don't prioritize regressions over bugs and features. Prioritization just using 'like' vote is quite worrying, do i need to write a bot to get attention on this 6 month old regression ? Community help didn't work too, since the proposed PR has not been merged since months, multiple comments about the impact of this regression didn't resonate too.

@YakDriver help would be appreciated, thanks.

[kai-ten]

Best thing you can do is give the PR #26746 a like, and hopefully Terraform finds it worthwhile to fix this broken aspect of their tool.

Also impacting my organization

[barthel]

@justinretzolk help would be appreciated, thanks.

[github-actions[bot]]

This functionality has been released in v4.59.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

[github-actions[bot]]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.