Cycle errors in 1.3.2

[jack-parsons-bjss]

Terraform Version

1.3.2

Terraform Configuration Files

Unable to provide due to volume

Debug Output

N/A

Expected Behavior

Terraform plans/applies as usual

Actual Behavior

In 1.3.2 a cycle is detected:

Error: Cycle: module.ous.aws_organizations_organizational_unit.level<redacted>["<redacted>"], module.ous.aws_organizations_organizational_unit.level<redacted>["<redacted>"], module.ous.aws_organizations_organizational_unit.level<redacted>["<redacted>"], module.ous.aws_organizations_organizational_unit.level<redacted>["<redacted>"], module.ous.aws_organizations_organizational_unit.level<redacted>["<redacted>"], module.ous.aws_organizations_organizational_unit.level<redacted>["<redacted>"], module.bs_gr_audit.module.lambdacron_remove_shield.aws_iam_role.main (destroy), module.bs_gr_shared.module.lambdacron_remove_shield.aws_iam_role.main (destroy), module.bs_gr_audit.aws_cloudwatch_event_rule.ec2_deletetags[0] (destroy), module.bs_gr_audit.module.lambdacron_remove_shield.aws_sns_topic_policy.main[0] (destroy), module.bs_gr_audit.module.lambdacron_remove_shield.aws_sns_topic.main[0] (destroy), module.bs_gr_audit.module.lambdacron_remove_shield.module.kms.aws_kms_key.main (destroy), module.bs_gr_shared.aws_cloudwatch_event_rule.elasticloadbalancing_removetags[0] (destroy), module.bs_gr_shared.module.lambdacron_remove_shield.module.kms.aws_iam_policy.user (destroy), module.bs_gr_audit.module.lambdacron_remove_shield.module.kms.aws_iam_policy.admin (destroy), module.bs_gr_audit.module.lambdacron_remove_shield.module.kms.aws_iam_policy.user (destroy), aws_organizations_organization.main, module.ous.var.organization_root_id (expand), module.bs_gr_audit.aws_lambda_permission.cloudtrail_delivery[0] (destroy), module.bs_gr_shared.module.lambdacron_remove_shield.module.kms.aws_iam_policy.admin (destroy), module.bs_gr_audit.module.lambdacron_remove_shield.module.kms.aws_kms_alias.main (destroy), module.bs_gr_audit.module.lambdacron_remove_shield.aws_lambda_function.main (destroy), module.bs_gr_audit.aws_lambda_permission.remove_shield_ec2_deletetags[0] (destroy), module.bs_gr_audit.aws_cloudwatch_event_target.ec2_deletetags[0] (destroy), module.bs_gr_audit.module.lambdacron_remove_shield.aws_cloudwatch_log_group.main (destroy), module.bs_gr_shared.module.lambdacron_remove_shield.aws_iam_role_policy_attachment.lambda_execution (destroy), module.bs_gr_shared.module.lambdacron_remove_shield.aws_iam_policy.lambda_execution (destroy), module.bs_gr_shared.aws_cloudwatch_event_rule.ec2_deletetags[0] (destroy), module.bs_gr_network.aws_cloudwatch_event_rule.ec2_deletetags[0] (destroy), module.bs_gr_network.module.lambdacron_remove_shield.aws_iam_role.main (destroy), module.bs_gr_network.module.lambdacron_remove_shield.module.kms.aws_iam_policy.admin (destroy), module.bs_gr_network.module.lambdacron_remove_shield.aws_iam_policy.lambda_execution (destroy), module.bs_gr_network.module.lambdacron_remove_shield.module.kms.aws_iam_policy.user (destroy), module.bs_gr_network.module.lambdacron_remove_shield.aws_iam_role_policy_attachment.lambda_execution (destroy), module.bs_gr_network.aws_cloudwatch_event_rule.elasticloadbalancing_removetags[0] (destroy), module.bs_gr_network.aws_lambda_permission.cloudtrail_delivery[0] (destroy), module.bs_gr_network.module.lambdacron_remove_shield.aws_lambda_function.main (destroy), module.bs_gr_network.aws_lambda_permission.remove_shield_ec2_deletetags[0] (destroy), module.bs_gr_network.module.lambdacron_remove_shield.aws_sns_topic.main[0] (destroy), module.bs_gr_network.aws_cloudwatch_event_target.ec2_deletetags[0] (destroy), module.bs_gr_network.aws_cloudwatch_event_target.elasticloadbalancing_removetags[0] (destroy), module.bs_gr_network.module.lambdacron_remove_shield.module.kms.aws_kms_alias.main (destroy), module.bs_gr_network.module.lambdacron_remove_shield.aws_sns_topic_policy.main[0] (destroy), module.bs_gr_network.module.lambdacron_remove_shield.module.kms.aws_kms_key.main (destroy), aws_organizations_account.network, provider["registry.terraform.io/hashicorp/aws"].network, module.bs_gr_network.aws_lambda_permission.remove_shield_elasticloadbalancing_removetags[0] (destroy), module.bs_gr_network.module.lambdacron_remove_shield.aws_cloudwatch_log_group.main (destroy), module.bs_gr_audit.aws_cloudwatch_event_target.elasticloadbalancing_removetags[0] (destroy), module.bs_gr_audit.aws_lambda_permission.remove_shield_elasticloadbalancing_removetags[0] (destroy), module.bs_gr_audit.aws_cloudwatch_event_rule.elasticloadbalancing_removetags[0] (destroy), module.bs_gr_shared.module.lambdacron_remove_shield.aws_cloudwatch_log_group.main (destroy), module.bs_gr_shared.module.lambdacron_remove_shield.module.kms.aws_kms_alias.main (destroy), module.bs_gr_shared.module.lambdacron_remove_shield.aws_sns_topic_policy.main[0] (destroy), module.bs_gr_shared.aws_lambda_permission.remove_shield_elasticloadbalancing_removetags[0] (destroy), module.bs_gr_shared.aws_cloudwatch_event_target.ec2_deletetags[0] (destroy), module.bs_gr_shared.module.lambdacron_remove_shield.aws_lambda_function.main (destroy), module.bs_gr_shared.aws_cloudwatch_event_target.elasticloadbalancing_removetags[0] (destroy), module.bs_gr_shared.module.lambdacron_remove_shield.aws_sns_topic.main[0] (destroy), module.bs_gr_shared.aws_lambda_permission.remove_shield_ec2_deletetags[0] (destroy), module.bs_gr_shared.module.lambdacron_remove_shield.module.kms.aws_kms_key.main (destroy), module.bs_gr_security.aws_cloudwatch_event_rule.ec2_deletetags[0] (destroy), module.bs_gr_security.module.lambdacron_remove_shield.aws_iam_policy.lambda_execution (destroy), module.bs_gr_security.module.lambdacron_remove_shield.aws_iam_role_policy_attachment.lambda_execution (destroy), module.bs_gr_security.module.lambdacron_remove_shield.aws_iam_role.main (destroy), module.bs_gr_security.aws_cloudwatch_event_rule.elasticloadbalancing_removetags[0] (destroy), module.bs_gr_security.module.lambdacron_remove_shield.aws_cloudwatch_log_group.main (destroy), module.bs_gr_security.aws_lambda_permission.cloudtrail_delivery[0] (destroy), module.bs_gr_security.module.lambdacron_remove_shield.module.kms.aws_iam_policy.user (destroy), module.bs_gr_security.module.lambdacron_remove_shield.aws_sns_topic_policy.main[0] (destroy), module.bs_gr_security.module.lambdacron_remove_shield.aws_lambda_function.main (destroy), module.bs_gr_security.aws_cloudwatch_event_target.elasticloadbalancing_removetags[0] (destroy), module.bs_gr_security.module.lambdacron_remove_shield.aws_sns_topic.main[0] (destroy), module.bs_gr_security.aws_lambda_permission.remove_shield_ec2_deletetags[0] (destroy), module.bs_gr_security.aws_lambda_permission.remove_shield_elasticloadbalancing_removetags[0] (destroy), module.bs_gr_security.module.lambdacron_remove_shield.module.kms.aws_kms_alias.main (destroy), module.bs_gr_security.aws_cloudwatch_event_target.ec2_deletetags[0] (destroy), module.bs_gr_security.module.lambdacron_remove_shield.module.kms.aws_kms_key.main (destroy), aws_organizations_account.security, provider["registry.terraform.io/hashicorp/aws"].security, module.bs_gr_security.module.lambdacron_remove_shield.module.kms.aws_iam_policy.admin (destroy), module.bs_gr_audit.module.lambdacron_remove_shield.aws_iam_role_policy_attachment.lambda_execution (destroy), module.bs_gr_audit.module.lambdacron_remove_shield.aws_iam_policy.lambda_execution (destroy), module.ous.aws_organizations_organizational_unit.level<redacted>["<redacted>"], module.ous.local.ous (expand), module.ous.output.ous (expand), aws_organizations_account.shared, provider["registry.terraform.io/hashicorp/aws"].shared, module.bs_gr_shared.aws_lambda_permission.cloudtrail_delivery[0] (destroy), aws_organizations_account.audit, provider["registry.terraform.io/hashicorp/aws"].audit

Thought to be resolved in 1.3.2 from #31843 however is still occurring. Downgrading to 1.2.9 causes this same configuration to succeed.

Steps to Reproduce

1. terraform apply

Additional Context

N/A

References

#31843

[jbardin]

Hi @jack-parsons-bjss,

Thanks for filing the issue. Without an example we unfortunately are not going to be able to diagnose the problem, or validate any solutions. If you could generate a standalone reproduction, it would help greatly. The next v1.3.4 release will have some more improvements here which may help if you have the ability to test with the current development branch.

One thing I can tell from the given cycle output, is that you have multiple providers which depend on managed resources in the same configuration. This is not fully supported (It's mentioned that you cannot safely access managed resource attributes from provider configuration in the docs here). This type of configuration also inherently causes cycles in some situations, which Terraform tries to detect and avoid, but there is not yet a universal mechanism to do this. You can see the issue #30465 showing the same problem from earlier versions too.

I also just mentioned in the linked issue that most of the known cases of these cycles should be resolved, so it would be very helpful to get an example here if the next release does not remedy the situation.

Thanks!

[WilliamABradley]

We are getting the same issue too on the 1.3x (1.3.2, 1.3.3) releases:

Error: Cycle: module.aws.module.org_medview.aws_organizations_account.environments["production"], module.aws.module.org_medview.local.account_master_role_arns (expand), module.aws.module.org_medview.output.account_master_role_arns (expand), module.aws.provider["registry.terraform.io/hashicorp/aws"].medview_production, module.aws.module.zone_medvieweducation_org.aws_route53_record.cname_records["em2636"] (destroy), module.aws.module.org_medview.aws_organizations_organizational_unit.unit, module.aws.module.org_medview.aws_organizations_account.environments["staging"]

We use dynamic providers in the current approach, as we dynamically provision AWS Subaccounts. This works on Terraform 1.2x.

Our approach is necessary for the lack of https://github.com/hashicorp/terraform/issues/25244 Lol, didn't even remember I created that issue!

[nwsparks]

Happening to us as well. Works on 1.2.4, does not work on 1.3.1, 1.3.3.

Attempting to track it down via graph nothing ends up being circled red.

It's occurring during plans that result in resources being removed...specifically in our case the plan is attempting to remove all the resources that exist in a module

module "stuff" {
  for_each = {stuff}
  ...
}

to

module "stuff" {
  for_each = {}
  ...
}

Error: Cycle:
module.eks.aws_iam_role_policy_attachment.policies["arn:aws:iam::aws:policy/AmazonEKSServicePolicy"],
module.eks.aws_kms_key.eks,
module.eks.aws_iam_role_policy_attachment.policies["arn:aws:iam::aws:policy/AmazonEKSClusterPolicy"],
module.security_groups.aws_security_group.allow_jenkins,
module.security_groups.output.allow_jenkins_id (expand),
module.security_groups.aws_security_group.eks_master,
module.security_groups.output.eks_master_id (expand),
module.eks.var.master_security_group_ids (expand),
module.eks.aws_iam_role.eks_service,
module.airflow_eks_resources["name"].module.airflow_namespace.kubernetes_secret.image_pull_secrets["docker-cfg"] (destroy),
module.airflow_eks_resources["name"].module.airflow_namespace.kubernetes_namespace.this (destroy),
module.airflow_eks_resources["name"].module.airflow_namespace.kubernetes_default_service_account.this (destroy),
module.eks.output.cluster_name (expand),
module.eks.output.certificate_authority (expand),
module.eks.output.endpoint (expand),
provider["registry.terraform.io/hashicorp/kubernetes"],
module.airflow_eks_resources["name"].kubernetes_persistent_volume_claim.claim (destroy),
module.airflow_eks_resources["name"].module.airflow_namespace.aws_eks_fargate_profile.this[0] (destroy),
module.eks.aws_cloudwatch_log_group.log_group,
module.eks.aws_eks_cluster.eks

[nwsparks]

@jbardin i just tested with the v1.3 branch and also main and the problem goes away in both.

[WilliamABradley]

I can confirm that 1.3.4 that was just release has fixed it on our side! 🙂

[jbardin]

Hello,

Without any examples or configuration to reproduce the problem, there's not going to be much else we can do here. As others have pointed out, known cases have been resolved by the latest release, so I'm going to close this out as complete for now.

Thanks!

[github-actions[bot]]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.