terraform init fails with conflicting module version requirements for hashicorp/google and hashicorp/google-beta

[danscarf]

TL;DR

• terraform init fails with conflicting module version requirements
• Affected providers are hashicorp/google and hashicorp/google-beta
• Trying to get the TF Cloud path working (not CloudBuild or Jenkins)

Expected behavior

• Terraform init command should complete without error

Observed behavior

 ~/src/github/terraform-google-modules/gcp-bootstrap/envs/shared  plan ?1  terraform init
Initializing the backend...
Initializing modules...

- Installed hashicorp/kubernetes v2.26.0 (signed by HashiCorp)

 Error: Failed to query available provider packages

 Could not retrieve the list of available versions for provider hashicorp/google-beta: no available releases match the given constraints >= 3.1.0, >= 3.43.0, >=
 3.50.0, >= 3.67.0, >= 3.77.0, ~> 4.5, >= 4.11.0, >= 4.28.0, >= 4.40.0, >= 4.64.0, < 5.0.0, >= 5.6.0, >= 5.9.0, < 6.0.0

 Error: Failed to query available provider packages

 Could not retrieve the list of available versions for provider hashicorp/google: no available releases match the given constraints >= 3.33.0, >= 3.43.0, >= 3.50.0,
│ >= 3.53.0, >= 3.67.0, >= 3.77.0, >= 3.83.0, >= 4.3.0, ~> 4.5, >= 4.25.0, >= 4.28.0, != 4.31.0, >= 4.40.0, >= 4.64.0, < 5.0.0, >= 5.6.0, >= 5.9.0, < 6.0.0

Terraform Configuration

I only modified the terraform.tfvars file and can't include those sensitive values.

 ~/src/github/terraform-google-modules  ls -l
total 0
drwxr-xr-x   5 dan  staff  160 Mar  2 14:26 gcp-bootstrap
drwxr-xr-x   5 dan  staff  160 Mar  2 11:47 gcp-environments
drwxr-xr-x   5 dan  staff  160 Mar  2 11:47 gcp-networks
drwxr-xr-x   5 dan  staff  160 Mar  2 11:47 gcp-org
drwxr-xr-x   5 dan  staff  160 Mar  2 11:47 gcp-projects
drwxr-xr-x  26 dan  staff  832 Mar  2 08:47 terraform-example-foundation
 ~/src/github/terraform-google-modules 

 ~/src/github/terraform-google-modules/gcp-bootstrap  plan ?1  tree                                                                                             ✔
.
└── envs
    └── shared
        ├── Dockerfile
        ├── README-GitHub.md
        ├── README-GitLab.md
        ├── README-Jenkins.md
        ├── README-Terraform-Cloud.md
        ├── README.md
        ├── backend.tf.cloud.example
        ├── backend.tf.example
        ├── cb.tf.example
        ├── files
        │   └── private_key_example.png
        ├── github.tf.example
        ├── gitlab.tf.example
        ├── groups.tf
        ├── jenkins.tf.example
        ├── main.tf
        ├── modules
        │   ├── cb-private-pool
        │   │   ├── README.md
        │   │   ├── main.tf
        │   │   ├── network.tf
        │   │   ├── outputs.tf
        │   │   ├── variables.tf
        │   │   ├── versions.tf
        │   │   └── vpn_ha.tf
        │   ├── gitlab-oidc
        │   │   ├── main.tf
        │   │   ├── outputs.tf
        │   │   ├── variables.tf
        │   │   └── versions.tf
        │   ├── jenkins-agent
        │   │   ├── README.md
        │   │   ├── files
        │   │   │   └── jenkins_gce_startup_script.sh
        │   │   ├── main.tf
        │   │   ├── outputs.tf
        │   │   ├── variables.tf
        │   │   ├── versions.tf
        │   │   └── vpn_ha.tf
        │   ├── parent-iam-member
        │   │   ├── main.tf
        │   │   ├── variables.tf
        │   │   └── versions.tf
        │   ├── parent-iam-remove-role
        │   │   ├── main.tf
        │   │   ├── variables.tf
        │   │   └── versions.tf
        │   └── tfc-agent-gke
        │       ├── README.md
        │       ├── main.tf
        │       ├── outputs.tf
        │       ├── variables.tf
        │       └── versions.tf
        ├── onprem.md
        ├── outputs.tf
        ├── provider.tf
        ├── sa.tf
        ├── scripts
        │   ├── git_create_branches_helper.sh
        │   └── push-to-repo.sh
        ├── terraform.tfvars
        ├── terraform_cloud.tf
        ├── variables.tf
        └── versions.tf

13 directories, 54 files
 ~/src/github/terraform-google-modules/gcp-bootstrap  plan ?1 

Terraform Version

Terraform v1.7.4
on darwin_arm64

Additional information

output of terraform providers command:

 ~/s/github/terraform-google-modules/gcp-bootstrap/envs/shared  plan ?1  terraform providers                                                                  1 ✘

Providers required by configuration:
.
├── provider[registry.terraform.io/hashicorp/google] >= 3.50.0, != 4.31.0
├── provider[registry.terraform.io/hashicorp/tfe] 0.48.0
├── provider[registry.terraform.io/hashicorp/random]
├── provider[registry.terraform.io/hashicorp/google-beta]
├── module.required_group
│   ├── provider[registry.terraform.io/hashicorp/google-beta] >= 3.67.0, < 6.0.0
│   └── provider[registry.terraform.io/hashicorp/google] >= 3.67.0, < 6.0.0
├── module.seed_bootstrap
│   ├── provider[registry.terraform.io/hashicorp/google] >= 3.50.0, < 6.0.0
│   ├── provider[registry.terraform.io/hashicorp/random] >= 3.1.0
│   ├── module.enable_cross_project_service_account_usage
│       ├── provider[registry.terraform.io/hashicorp/google] >= 3.53.0, < 6.0.0
│       └── provider[registry.terraform.io/hashicorp/null] >= 2.1.0
│   ├── module.kms
│       └── provider[registry.terraform.io/hashicorp/google] >= 3.53.0, < 6.0.0
│   └── module.seed_project
│       ├── provider[registry.terraform.io/hashicorp/google-beta] >= 4.28.0, < 6.0.0
│       ├── provider[registry.terraform.io/hashicorp/google] >= 4.28.0, < 6.0.0
│       ├── module.project-factory
│           ├── provider[registry.terraform.io/hashicorp/random] >= 2.2.0
│           ├── provider[registry.terraform.io/hashicorp/time] >= 0.5.0
│           ├── provider[registry.terraform.io/hashicorp/google] >= 3.50.0, < 6.0.0
│           ├── provider[registry.terraform.io/hashicorp/google-beta] >= 3.50.0, < 6.0.0
│           ├── provider[registry.terraform.io/hashicorp/null] >= 2.1.0
│           └── module.project_services
│               ├── provider[registry.terraform.io/hashicorp/google] >= 3.43.0, < 6.0.0
│               └── provider[registry.terraform.io/hashicorp/google-beta] >= 3.43.0, < 6.0.0
│       ├── module.quotas
│           └── provider[registry.terraform.io/hashicorp/google-beta] >= 4.11.0, < 6.0.0
│       ├── module.shared_vpc_access
│           ├── provider[registry.terraform.io/hashicorp/google] >= 3.43.0, < 6.0.0
│           └── provider[registry.terraform.io/hashicorp/google-beta] >= 3.43.0, < 6.0.0
│       ├── module.budget
│           └── provider[registry.terraform.io/hashicorp/google] >= 4.28.0, < 6.0.0
│       ├── module.essential_contacts
│           ├── provider[registry.terraform.io/hashicorp/google-beta] >= 3.43.0, < 6.0.0
│           └── provider[registry.terraform.io/hashicorp/google] >= 3.43.0, < 6.0.0
│       └── module.gsuite_group
│           └── provider[registry.terraform.io/hashicorp/google] >= 3.43.0, < 6.0.0
├── module.seed_project_iam_member
│   ├── provider[registry.terraform.io/hashicorp/random]
│   ├── provider[registry.terraform.io/hashicorp/google] >= 3.77.0
│   └── provider[registry.terraform.io/hashicorp/google-beta] >= 3.77.0
├── module.optional_group
│   ├── provider[registry.terraform.io/hashicorp/google] >= 3.67.0, < 6.0.0
│   └── provider[registry.terraform.io/hashicorp/google-beta] >= 3.67.0, < 6.0.0
├── module.tfc-oidc
│   └── provider[registry.terraform.io/hashicorp/google] >= 3.53.0, < 5.0.0
├── module.parent_iam_member
│   ├── provider[registry.terraform.io/hashicorp/random]
│   ├── provider[registry.terraform.io/hashicorp/google] >= 3.77.0
│   └── provider[registry.terraform.io/hashicorp/google-beta] >= 3.77.0
├── module.cicd_project_iam_member
│   ├── provider[registry.terraform.io/hashicorp/google-beta] >= 3.77.0
│   ├── provider[registry.terraform.io/hashicorp/random]
│   └── provider[registry.terraform.io/hashicorp/google] >= 3.77.0
├── module.tfc_agent_gke
│   ├── provider[registry.terraform.io/hashicorp/random] >= 3.4.3, < 4.0.0
│   ├── provider[registry.terraform.io/hashicorp/google] >= 4.3.0, < 6.0.0
│   ├── provider[registry.terraform.io/hashicorp/kubernetes] >= 2.0.0, < 3.0.0
│   ├── provider[registry.terraform.io/hashicorp/google-beta]
│   ├── module.private_service_connect
│       ├── provider[registry.terraform.io/hashicorp/google-beta] >= 3.50.0, < 6.0.0
│       ├── provider[registry.terraform.io/hashicorp/google] >= 3.50.0, < 6.0.0
│       ├── module.googleapis
│           ├── provider[registry.terraform.io/hashicorp/google] >= 4.40.0, < 6.0.0
│           └── provider[registry.terraform.io/hashicorp/google-beta] >= 4.40.0, < 6.0.0
│       ├── module.pkg_dev
│           ├── provider[registry.terraform.io/hashicorp/google-beta] >= 4.40.0, < 6.0.0
│           └── provider[registry.terraform.io/hashicorp/google] >= 4.40.0, < 6.0.0
│       └── module.gcr
│           ├── provider[registry.terraform.io/hashicorp/google] >= 4.40.0, < 6.0.0
│           └── provider[registry.terraform.io/hashicorp/google-beta] >= 4.40.0, < 6.0.0
│   ├── module.tfc_agent_cluster
│       ├── provider[registry.terraform.io/hashicorp/kubernetes] ~> 2.10
│       ├── provider[registry.terraform.io/hashicorp/random] >= 2.1.0
│       ├── provider[registry.terraform.io/hashicorp/google] >= 5.9.0, < 6.0.0
│       └── provider[registry.terraform.io/hashicorp/google-beta] >= 5.9.0, < 6.0.0
│   ├── module.hub
│       ├── provider[registry.terraform.io/hashicorp/google] >= 5.6.0, < 6.0.0
│       └── provider[registry.terraform.io/hashicorp/google-beta] >= 5.6.0, < 6.0.0
│   └── module.network
│       ├── provider[registry.terraform.io/hashicorp/google-beta] >= 4.64.0, < 6.0.0
│       ├── provider[registry.terraform.io/hashicorp/google] >= 4.64.0, < 6.0.0
│       ├── module.routes
│           └── provider[registry.terraform.io/hashicorp/google] >= 3.83.0, < 6.0.0
│       ├── module.subnets
│           └── provider[registry.terraform.io/hashicorp/google] >= 4.25.0, < 6.0.0
│       ├── module.vpc
│           ├── provider[registry.terraform.io/hashicorp/google-beta] >= 4.64.0, < 6.0.0
│           └── provider[registry.terraform.io/hashicorp/google] >= 4.64.0, < 6.0.0
│       └── module.firewall_rules
│           └── provider[registry.terraform.io/hashicorp/google] >= 3.33.0, < 6.0.0
├── module.org_iam_member
│   ├── provider[registry.terraform.io/hashicorp/google] >= 3.77.0
│   ├── provider[registry.terraform.io/hashicorp/google-beta] >= 3.77.0
│   └── provider[registry.terraform.io/hashicorp/random]
├── module.tfc_cicd
│   ├── provider[registry.terraform.io/hashicorp/google-beta] ~> 4.5
│   ├── provider[registry.terraform.io/hashicorp/google] ~> 4.5
│   ├── module.project-factory
│       ├── provider[registry.terraform.io/hashicorp/time]
│       ├── provider[registry.terraform.io/hashicorp/google] >= 3.50.0, < 5.0.0
│       ├── provider[registry.terraform.io/hashicorp/google-beta] >= 3.50.0, < 5.0.0
│       ├── provider[registry.terraform.io/hashicorp/null] >= 2.1.0
│       ├── provider[registry.terraform.io/hashicorp/random] >= 2.2.0
│       └── module.project_services
│           ├── provider[registry.terraform.io/hashicorp/google-beta] >= 3.43.0, < 5.0.0
│           └── provider[registry.terraform.io/hashicorp/google] >= 3.43.0, < 5.0.0
│   ├── module.quotas
│       └── provider[registry.terraform.io/hashicorp/google-beta] >= 3.1.0, < 5.0.0
│   ├── module.shared_vpc_access
│       ├── provider[registry.terraform.io/hashicorp/google] >= 3.43.0, < 5.0.0
│       └── provider[registry.terraform.io/hashicorp/google-beta] >= 3.43.0, < 5.0.0
│   ├── module.budget
│       └── provider[registry.terraform.io/hashicorp/google] ~> 4.5
│   ├── module.essential_contacts
│       ├── provider[registry.terraform.io/hashicorp/google] >= 3.43.0, < 5.0.0
│       └── provider[registry.terraform.io/hashicorp/google-beta] >= 3.43.0, < 5.0.0
│   └── module.gsuite_group
│       └── provider[registry.terraform.io/hashicorp/google] >= 3.43.0, < 5.0.0
└── module.bootstrap_projects_remove_editor
    └── provider[registry.terraform.io/hashicorp/google] >= 3.77.0

 ~/src/github/terraform-google-modules/gcp-bootstrap/envs/shared  plan ?1 

[danscarf]

The dependency conflict is from the tfc_agent_gke module.

This depends on google-beta versions >= 5.6.0, which creates a conflict with several other dependencies of this module which constrain the version below 5.0.0.

My workaround was to comment out the module definition, as the init pulls down the modules even if enable_tfc_cloud_agents = false.

# module "tfc_agent_gke" {
#   source = "./modules/tfc-agent-gke"
#   count  = var.enable_tfc_cloud_agents == true ? 1 : 0

#   project_id             = module.tfc_cicd.project_id
#   project_number         = module.tfc_cicd.project_number
#   tfc_agent_token        = tfe_agent_token.tfc_agent_token[0].token
#   create_service_account = false
#   service_account_email  = google_service_account.terraform-env-sa["bootstrap"].email
#   service_account_id     = google_service_account.terraform-env-sa["bootstrap"].id

#   //If you are using Terraform Cloud Agents, un-comment this block after the first apply according README instructions
#   # providers = {
#   #   kubernetes = kubernetes
#   # }
# }

Also, there hasn't been any activity on this issue for about a week. Did I miss a step in filing the issue? Thank you!

[github-actions[bot]]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 7 days

[daniel-cit]

depends of https://github.com/GoogleCloudPlatform/terraform-google-tf-cloud-agents/pull/24

[eeaton]

@apeabody Can you review and merge https://github.com/GoogleCloudPlatform/terraform-google-tf-cloud-agents/pull/24 ? We should be able to close this issue once that is complete