Switching from `databrickslabs` to `databricks` namespace

[nfx]

trafficstars

To make Databricks Terraform Provider generally available, we've moved it from https://github.com/databrickslabs to https://github.com/databricks. We've worked closely with the Terraform Registry team at Hashicorp to ensure a smooth migration. Existing terraform deployments continue to work as expected without any action from your side. We ask you to replace databrickslabs/databricks with databricks/databricks in all your .tf files.

You should have .terraform.lock.hcl file in your state directory that is checked into source control. terraform init will give you the following warning.

Warning: Additional provider information from registry 

The remote registry returned warnings for registry.terraform.io/databrickslabs/databricks:
- For users on Terraform 0.13 or greater, this provider has moved to databricks/databricks. Please update your source in required_providers.

After you replace databrickslabs/databricks with databricks/databricks in the required_providers block, the warning will disappear. Do a global "search and replace" in *.tf files. Alternatively you can run python3 -c "$(curl -Ls https://dbricks.co/updtfns)" from the command-line, that would do all the boring work for you.

If you didn't check-in .terraform.lock.hcl to the source code version control, you may you may see Failed to install provider error. Please follow the simple steps described in the troubleshooting guide.

[yewsiang]

Hi there, we're using the cloud backend and did not check-in .terraform.lock.hcl to version control.

terraform {
  required_providers {
    databricks = {
      source = "databricks/databricks"
    }
  }
  cloud {
    organization = "X"

    workspaces {
      name = "Y"
    }
  }
}

While following instructions in the troubleshooting guide, we've replaced all databrickslab/databricks with databricks/databricks in the TF files but running terraform state replace-provider databrickslabs/databricks databricks/databricks throws the following error: No state file was found!. I don't think specifying -state will work since it's meant for local states?

It seems like terraform state replace-provider does not work well with cloud states?

Would appreciate assistance!

[JCZuurmond]

🚨 Yesterday the Terraform PR CI started breaking with a Failed to install provider message. I reran the CI, maybe the error is caused by a network failure, the second run fails with the same issue. The PR does not contain changes that I expect to cause this error, I run the CI for the main branch, also the same issue.

👀 I go to the the Databricks Terraform provider Github repo (here), see a three hours old release. I click the release, the release message explains the issue I am having, why this issue occurs now and provides a link to a trouble shooting guide. There is also a Github issue (this issue) that explains the problem, the issue is pinned so that it is easy to find.

🏗️ The trouble shooting guide contains three one-command steps - two if you exclude the verification step - to resolve the issue. For one step you wrote a script to help us resolve the issue easier.

💡 On top of all this, I learn something new, about why there is a .terraform.lock.hcl.

🙏 This is the best Terraform provider that I know. One of the best maintained open-source projects that I know. @nfx, you are awesome! Thank you for making my life a little easier.

🎉 I wanted to take the time to thank you. The issue is resolved, the CI is running smoothly again. Thanks!

[stevenwhayes]

When following the update instructions, with a checked in .terraform.lock.hcl file, my plan is erroring and complaining about orphaned objects for users. Example: Error: Provider configuration not present │ │To work with │module.test_project_databricks_workspace_config[0].databricks_user.admins["redacted"] │(orphan) its original provider configuration at │provider["registry.terraform.io/databrickslabs/databricks"].testproj_new is │required, but it has been removed. This occurs when a provider │configuration is removed while objects created by that provider still exist │in the state. Re-add the provider configuration to destroy │module.test_project_databricks_workspace_config[0].databricks_user.admins["redacted"] │(orphan), after which you can remove the provider configuration again.

Does this mean that we have to run the terraform state replace-provider databrickslabs/databricks databricks/databricks on our states?

[nfx]

@stevenwhayes yes, sometimes you have to update the state before proceeding. Please try it out and confirm the steps - I’ll update the error troubleshooting guide

[stack72]

@nfx please can you also change the go.mod namespace here? I am not able to pull this into pulumi until this is done as there's a mismatch between the repo and the gomodule

[nfx]

@stack72 of course, that change will be there soon

[janaekj]

In versions.tf of root module we had databricks profile set to match named profile from .databrickscfg. after the upgrade to databricks/databricks it now errors as its looking and expecting some [DEFAULT] profile. adding [DEFAULT] to .databrickscfg to match the expected named profile seems to be valid workaround. provider "databricks" { profile = var.environment }

replacing existing profile in databrickscfg with [DEFAULT] errored as it is still looking for the custom named profile in addition to [DEFAULT].

[nfx]

@janaekj required_providers block has nothing to do with provider block

[barywhyte]

@nfx Can we say that the migration of databricks namespace to databricks/databricks needs to done for development, staging and production environments at the same time, especially in situation where several terraform modules are being simultaneously shared among these three environments?

My observation is that an attempt to try this process in dev appears to break my staging and production environments with the below error:

Error: Failed to install provider

Error while installing databrickslabs/databricks v1.0.1: checksum list has no
SHA-256 hash for
"https://github.com/databricks/terraform-provider-databricks/releases/download/v1.0.1/terraform-provider-databricks_1.0.1_darwin_amd64.zip"

My intention is to try this out in dev first before moving to staging and prod but it looks like this is impossible to do

[nfx]

@barywhyte it's just replacing the provider coordinates. binaries didn't change.

[RKSelvi]

We have terraform state file in azure storage account (remote backend) and after upgrading to databricks version with 1.0.2, I had to update the provider. Updating the terraform state file with these commands not working. I manually updated terraform state file and that works. But we cannot update terraform state file in Prod

    terraform init -input=false -upgrade -backend-config=${{ github.workspace }}/.tfbackend/${{ github.workflow }}
    terraform state replace-provider -auto-approve databrickslabs/databricks databricks/databricks
    terraform init -input=false -upgrade -backend-config=${{ github.workspace }}/.tfbackend/${{ github.workflow }}
    terraform plan -input=false -out=tfplan
    terraform apply -auto-approve tfplan

Getting this error: Initializing the backend... Successfully configured the backend "azurerm"! Terraform will automatically use this backend unless the backend configuration changes. Initializing provider plugins...

• Finding latest version of databrickslabs/databricks...
• Finding databricks/databricks versions matching "1.0.2"...
• Finding hashicorp/azurerm versions matching "~> 3.14.0"...
• Finding hashicorp/azuread versions matching "~> 2.26.1"...
• Installing databrickslabs/databricks v1.0.2...
• Installing databricks/databricks v1.0.2...
• Installed databricks/databricks v1.0.2 (self-signed, key ID 92A95A66446BCE3F)
• Installing hashicorp/azurerm v3.14.0...
• Installed hashicorp/azurerm v3.14.0 (signed by HashiCorp)
• Installing hashicorp/azuread v2.26.1...
• Installed hashicorp/azuread v2.26.1 (signed by HashiCorp) Partner and community providers are signed by their developers. If you'd like to know more about provider signing, you can read about it here: https://www.terraform.io/docs/cli/plugins/signing.html ╷ │Error: Failed to install provider │ │Error while installing databrickslabs/databricks v1.0.2: checksum list has │no SHA-256 hash for │"https://github.com/databricks/terraform-provider-databricks/releases/download/v1.0.2/terraform-provider-databricks_1.0.2_linux_amd64.zip" ╵ ╷ │Warning: Additional provider information from registry │ │The remote registry returned warnings for │registry.terraform.io/databrickslabs/databricks: │- For users on Terraform 0.13 or greater, this provider has moved to │databricks/databricks. Please update your source in required_providers

[nfx]

@RKSelvi make sure you have .terraform.lock.hcl in the source control. This commonly fixes all the issues.

[RKSelvi]

@RKSelvi make sure you have .terraform.lock.hcl in the source control. This commonly fixes all the issues.

@nfx Tried but not working. I could be doing something wrong! As I did not have hcl file, upload the hcl from wf pipeline artifact, downloaded and added to the repo and checked in. Still the same error. The TF init is trying to install databrickslabs/databricks version 1.1.0 ( - Installing databrickslabs/databricks v1.1.0...). The version I had before upgrading was 0.5.9. I also tried removing the version of databrickslabs/databricks in the .terraform.lock.hcl file hoping that a newer version will be updated.

provider.tf: terraform { required_providers { azurerm = "~> 3.15.0" azuread = "~> 2.26.1" databricks = { source = "databricks/databricks" version = "1.1.0" } }

backend "azurerm" { } } provider "azurerm" { skip_provider_registration = true features {} } provider "azuread" {}

Failure: Initializing the backend... Successfully configured the backend "azurerm"! Terraform will automatically use this backend unless the backend configuration changes. Initializing provider plugins...

• Finding databricks/databricks versions matching "1.1.0"...
• Finding hashicorp/azurerm versions matching "~> 3.15.0"...
• Finding hashicorp/azuread versions matching "~> 2.26.1"...
• Finding latest version of databrickslabs/databricks...
• Installing databrickslabs/databricks v1.1.0...
• Installing databricks/databricks v1.1.0...
• Installed databricks/databricks v1.1.0 (self-signed, key ID 92A95A66446BCE3F)
• Installing hashicorp/azurerm v3.15.0...
• Installed hashicorp/azurerm v3.15.0 (signed by HashiCorp)
• Installing hashicorp/azuread v2.26.1...
• Installed hashicorp/azuread v2.26.1 (signed by HashiCorp) Partner and community providers are signed by their developers. If you'd like to know more about provider signing, you can read about it here: https://www.terraform.io/docs/cli/plugins/signing.html ╷ │Warning: Additional provider information from registry ╷ │Error: Failed to install provider │ │Error while installing databrickslabs/databricks v1.1.0: checksum list has │no SHA-256 hash for │"https://github.com/databricks/terraform-provider-databricks/releases/download/v1.1.0/terraform-provider-databricks_1.1.0_linux_amd64.zip" ╵ │ │The remote registry returned warnings for │registry.terraform.io/databrickslabs/databricks: │- For users on Terraform 0.13 or greater, this provider has moved to │databricks/databricks. Please update your source in required_providers. ╵

[RKSelvi]

So I updated tf state file manually and let the workflow run and now the hcl is updated with databricks/databricks 1.1.0 version. So looks like terraform init is not able to update it and fails for databrickslabs/databricks even with .terraform.lock.hcl checked in. Not sure what is going on....

[nfx]

@RKSelvi i think you need to add provider hashes of both used and the most recent versions to .terraform.lock.hcl

[barywhyte]

@nfx , Just curious, do we need to be on terraform version 0.14+ for us to have successful migration of databricks provider namespace? I asked because currently in our configuration, we don't have .terraform.lock.hcl file and we are using terraform version 0.13 and from this https://www.terraform.io/language/files/dependency-lock, .terraform.lock.hcl only seems to apply to terraform version 0.14+ only. I find this a bit confusing and want to know if terraform versions have anything (however small) to do with this switching to the new databricks namespace.

[RKSelvi]

@RKSelvi i think you need to add provider hashes of both used and the most recent versions to .terraform.lock.hcl

This is not working. I added both versions to .terraform.lock.hcl but terraform init fails as the tf state file still has databrickslabs/databricks. The only fix I have it working if I need to move forward with databricks latest version and provider update by updating the tf state file directly/manually in Azure portal.

[nfx]

@barywhyte the share of provider adopters using tf v0.13, v0.12, and v0.11 is extremely low, hence I strongly recommend updating to the latest possible version. Approximately 80% of adopters are using tf 1.0+

provider functionality itself shouldn't be tied to any specific tf version - i know couple of working v0.11 deployments, for example. it's only about the Terraform Registry and the way how provider is auto-discovered.

[nfx]

@RKSelvi perhaps http://discuss.hashicorp.com/ might be able to answer your question.

[barywhyte]

When following the update instructions, with a checked in .terraform.lock.hcl file, my plan is erroring and complaining about orphaned objects for users. Example: Error: Provider configuration not present │ │To work with │module.test_project_databricks_workspace_config[0].databricks_user.admins["redacted"] │(orphan) its original provider configuration at │provider["registry.terraform.io/databrickslabs/databricks"].testproj_new is │required, but it has been removed. This occurs when a provider │configuration is removed while objects created by that provider still exist │in the state. Re-add the provider configuration to destroy │module.test_project_databricks_workspace_config[0].databricks_user.admins["redacted"] │(orphan), after which you can remove the provider configuration again.

Does this mean that we have to run the terraform state replace-provider databrickslabs/databricks databricks/databricks on our states?

@nfx I observed that running terraform state replace-provider databrickslabs/databricks databricks/databricks in a git feature branch, present a challenge for other engineers branching off of master with the above error. For me, this error still occurred even after running state replace-provider command and having .terraform.lock.hcl file. Not sure if there is something I am missing or not doing properly here. Do we need to make some sort of changes to .terraform.lock.hcl ?

[barywhyte]

@barywhyte the share of provider adopters using tf v0.13, v0.12, and v0.11 is extremely low, hence I strongly recommend updating to the latest possible version. Approximately 80% of adopters are using tf 1.0+

provider functionality itself shouldn't be tied to any specific tf version - i know couple of working v0.11 deployments, for example. it's only about the Terraform Registry and the way how provider is auto-discovered.

Thank you for this!

[barywhyte]

All issues now resolved from my end. Had to apply terraform state replace-provider databrickslabs/databricks databricks/databricks command across my three deployment environments (dev, staging, prod) at the same time from a feature branch before finally merging to master branch.

That was the solution to :

Error: Provider configuration not present │ │To work with │module.test_project_databricks_workspace_config[0].databricks_user.admins["redacted"] │(orphan) its original provider configuration at │provider["registry.terraform.io/databrickslabs/databricks"].testproj_new is │required, but it has been removed. This occurs when a provider │configuration is removed while objects created by that provider still exist │in the state. Re-add the provider configuration to destroy │module.test_project_databricks_workspace_config[0].databricks_user.admins["redacted"] │(orphan), after which you can remove the provider configuration again.

error