Need instructions for migrating from hashicorp/github to integrations/github

[docwhat]

Now that GitHub has taken over ownership of the github provider we need some instructions for upgrading.

I tried the obvious:

1. update required_providers:
   1. change source attribute to integrations/github.
   2. change version to 4.2.0 (from 4.1.0 in my case)
2. terraform state replace-provider registry.terraform.io/hashicorp/github registry.terraform.io/integrations/github
3. terraform init
   • This shows it installing both hashicorp 4.1.0 and integrations 4.2.0
4. terraform plan

Unfortunately, it then seems to hang forever.

TRACE shows that it is trying to access github.com (it should be using our enterprise server) without specifying the org using the hashicorp 4.1.0 provider:

[DEBUG] plugin.terraform-provider-github_v4.1.0_x4: GET /orgs//teams/sre-team HTTP/1.1
[DEBUG] plugin.terraform-provider-github_v4.1.0_x4: Host: api.github.com

There is exactly one GET using the 4.2 provider:

[DEBUG] plugin.terraform-provider-github_v4.2.0: GET /api/v3/orgs/the-org-on-our-ghe HTTP/1.1
[DEBUG] plugin.terraform-provider-github_v4.2.0: Host: github.example.com

No matter what I do, terraform insists on using the official hashicorp provider.

$ terraform -version
Terraform v0.14.4
+ provider registry.terraform.io/hashicorp/github v4.1.0
+ provider registry.terraform.io/integrations/github v4.2.0

[frbayart]

On my side I make:

1. edit all .TF files (plan and modules too)
2. terraform state replace-provider kensu.io/kensu/github integrations/github
3. terraform init
4. terraform plan

And it works; at the beginning I didn't update my modules so I got some confusion.... (kensu.io/kensu/github is fork of hashicorp/github with last features suggested on the repo)

[jcudit]

🤔 maybe we document this as a breaking change? I can get the CHANGELOG updated if it helps boost visibility of the state replace-provider procedure above.

[frbayart]

Also @docwhat if the problem persists, it's probably useful to check terraform providers

You have details per plan and modules, output is like:

Providers required by configuration:
.
├── provider[registry.terraform.io/integrations/github] 4.2.0
├── provider[registry.terraform.io/hashicorp/vault] ~> 2.11
├── module.dam-mock-module-webui
│   └── provider[registry.terraform.io/integrations/github] 4.2.0
|── module...
 
...

Providers required by state:

    provider[registry.terraform.io/hashicorp/vault]

    provider[registry.terraform.io/integrations/github]

[aliculPix4D]

@docwhat I am also seeing the same issue as you do and it seems that, at least for me, it was due to the bug described here: https://github.com/integrations/terraform-provider-github/issues/655#issuecomment-763394024 where both values for organization and token in provider block were ignored.

provider "github" {
  organization = "..."
  token        = var.github_token
}

by setting the GITHUB_TOKEN ,GITHUB_ORGANIZATION environmental variables, the upgrade was successful and only integrations/github (4.3.0) can be used.

[jcudit]

terraform {
  required_providers {
    github = {
      source = "integrations/github"
    }
  }
}

Adding the required_providers configuration block may unblock others experiencing this.

[MRostanski]

@jcudit no, unfortunately it does not. I have:

terraform {
  required_providers {
    github = {
      source  = "integrations/github"
      version = "~> 4.4.0"
    }
    random = {
      source  = "hashicorp/random"
      version = "~> 3.1.0"
    }
    tls = {
      source  = "hashicorp/tls"
      version = "~> 3.1.0"
    }
  }
}

and still both get installed:

Initializing provider plugins...

• Finding hashicorp/random versions matching "~> 3.1.0"...
• Finding hashicorp/tls versions matching "~> 3.1.0"...
• Finding latest version of hashicorp/github...
• Finding integrations/github versions matching "~> 4.4.0"...
• Installing hashicorp/random v3.1.0...
• Installed hashicorp/random v3.1.0 (signed by HashiCorp)
• Installing hashicorp/tls v3.1.0...
• Installed hashicorp/tls v3.1.0 (signed by HashiCorp)
• Installing hashicorp/github v4.4.0...
• Installed hashicorp/github v4.4.0 (signed by HashiCorp)
• Installing integrations/github v4.4.0...
• Installed integrations/github v4.4.0 (signed by a HashiCorp partner, key ID 38027F80D7FD5FB2)

Partner and community providers are signed by their developers. If you'd like to know more about provider signing, you can read about it here: https://www.terraform.io/docs/cli/plugins/signing.html

Terraform has created a lock file .terraform.lock.hcl to record the provider selections it made above. Include this file in your version control repository so that Terraform can guarantee to make the same selections by default when you run "terraform init" in the future.

Warning: Additional provider information from registry

The remote registry returned warnings for registry.terraform.io/hashicorp/github:

• For users on Terraform 0.13 or greater, this provider has moved to integrations/github. Please update your source in required_providers.

What is going on in terraform providers:

Providers required by configuration:
.
├── provider[registry.terraform.io/integrations/github] ~> 4.2.0
├── provider[registry.terraform.io/hashicorp/random] ~> 3.1.0
├── provider[registry.terraform.io/hashicorp/tls] ~> 3.1.0
├── module.github_repository
│   └── provider[registry.terraform.io/hashicorp/github]
├── module.naming_convention
│   └── provider[registry.terraform.io/hashicorp/random]
└── module.ssh_key
    └── provider[registry.terraform.io/hashicorp/tls]

[rmc47]

@MRostanski it looks like you're hitting the same thing we just did: if you use modules, each module needs to declare its own required_providers block with the source of the github module pointing to integrations/github instead of hashicorp/github.

Without that, you end up with the mix of the two providers...

[ops-hummus]

The workaround is to put this snippet with the main.tf and also inside the github module directory. terraform { required_providers { github = { source = "integrations/github" version = "4.13.0" } } }

lets assume we have main.tf and subdirectory containing github.tf. The main file uses a module that uses the github resource. Now the problem begins. Terraform download both integrations/github and hashicorp/github, but if you put the required_providers in both places, with the main and within the subdirectory, it stops the problem. Hope I was clear.

[aliculPix4D]

The workaround is to put this snippet with the main.tf and also inside the github module directory.

Another simplification that worked for me is to move the following section from main.tf to i.e tf-modules/github/main.tf

provider "github" {
  token = var.github_token
  owner = "somename"
}

and just send the token to the module as a variable:

module "github" {
  source = "../tf-modules/github"

  github_token  = var.github_token
}

this results in terraform providers command to output only :

This should prevent terraform to download hashicorp/github provider. Note that you also need to replace the provider in remote state file using terraform state replace-provider

[TheJackson]

@rmc47: @MRostanski it looks like you're hitting the same thing we just did: if you use modules, each module needs to declare its own required_providers block with the source of the github module pointing to integrations/github instead of hashicorp/github. Without that, you end up with the mix of the two providers...

This is the one, thanks for this workaround!

[larstobi]

I have already run terraform state replace-provider -- registry.terraform.io/hashicorp/github registry.terraform.io/integrations/github so there are no references to hashicorp/github in the state.

When using github resource inside modules, then this becomes quite confusing. As an experiment, I put nothing github under required_providers in the top level, nor inside the modules. When I then run terraform init, it wil pick a mix of hashicorp/github and integrations/github, but both with the same version (v4.28.0). In addition it prints a message from the Registry:

Warning: Additional provider information from registry

The remote registry returned warnings for registry.terraform.io/hashicorp/github:

• For users on Terraform 0.13 or greater, this provider has moved to integrations/github. Please update your source in required_providers.

Now, if I put integrations/github in required_providers in the top level terraform only, and nothing in the modules, then it still downloads a mix of the two provider names, and prints the above warning from the registry.

It initialized successfully, but the provider config in the top level isn't inherited to the module, so it can't work.

Then, if I integrations/github and provider config only into the modules and not in the top level, then it only downloads integrations/github and there is no mention of hashicorp/github. It initialized successfully and the provider config is correctly set.

So, I guess this problem is caused by both hashicorp/github and integrations/github using the same name github, and that if I specify no provider, it will by default pick hashicorp/github and in addition integrations/github if it's referenced in the state file. If it would always go for integrations/github instead, then maybe the whole problem goes away?

[ahmadnassri]

I can confirm @larstobi findings on my end.

[usmonster]

So, I guess this problem is caused by both hashicorp/github and integrations/github using the same name github, and that if I specify no provider, it will by default pick hashicorp/github and in addition integrations/github if it's referenced in the state file.

Can this be considered a bug, then, and not just about documentation?

[kfcampbell]

I'm not sure how I would go about troubleshooting that at the provider level. I wonder if that's something specific to the Terraform platform or tooling.

[jackyou84]

I meet same issue lately. I try to following steps:

a. change providers from module

required_providers {
    github = {
      source  = "integrations/github"
      version = "~> 5.17"
    }
  }

in local just refer providers:

provider "github" {
  owner = var.github_organization
  token = var.github_token
}

b. rm -rf .terraform .terraform.lock.hcl c. terraform init it still install hashicorp/github by default. try to relpace state terraform state replace-provider registry.terraform.io/hashicorp/github registry.terraform.io/integrations/github and run step b and c. but getting same result.

finally, I add following block in local tf. and run step b and c.

terraform {
  required_providers {
    github = {
      source = "integrations/github"
    }
  }
}

and only integrations/github is used. and tf plan also smoothly.

[jackyou84]

so seems to switch providers from old one. we must update both module and local tf provider.

[mado-m]

I have also solved this problem by using replace-provider.

terraform state replace-provider registry.terraform.io/hashicorp/github registry.terraform.io/integrations/github