Unable to deploy with a specific target name (working previously)

[arielhgl]

Describe the issue

I am trying to deploy the bundle (python-template) with preprod target name. I am able to validate the preprod and dev target and deploy dev target name without running into any issues, but when I try to deploy preprod it fails.

Error message is "Error: failed to read schema for [my_job_name] in registry.terraform.io/databricks/databricks: failed to instantiate provider "registry.terraform.io/databricks/databricks" to obtain schema: Unrecognized remote plugin message: This usually means that the plugin is either invalid or simply needs to be recompiled to support the latest protocol."

The issues are similar to [/issues/1613] but I am not able to fix with given suggestions.

I have no issues deploying the exact same command on other bundles, it just seems to fail under this the new created repo after updating to 2.5.0 VSCode extension. I have tried downgrading as well, but it seems there was some underlying change that the error is now persistent.

Additionally, when I change the Target name using the UI of the extension I get the error message:

Extension databricks.databricks has provided and invalid tree item.

Steps to reproduce the behavior

Create a new bundle with databricks bundle init default-python then run databricks bundle deploy then error.

Expected Behavior

bundle deploy results in "Deployment complete!"

Actual Behavior

bundle deploy results in "Error: failed to read schema for [my_job_name] in registry.terraform.io/databricks/databricks: failed to instantiate provider "registry.terraform.io/databricks/databricks" to obtain schema: Unrecognized remote plugin message: This usually means that the plugin is either invalid or simply needs to be recompiled to support the latest protocol."

OS and CLI version

Windows v0.236.0

Is this a regression?

Not sure.

Debug Logs

Error: failed to read schema for [my_job_name] in registry.terraform.io/databricks/databricks: failed to instantiate provider "registry.terraform.io/databricks/databricks" to obtain schema: Unrecognized remote plugin message: This usually means that the plugin is either invalid or simply needs to be recompiled to support the latest protocol.

[pietern]

Thanks for reporting.

Could you try to remove the .databricks directory in the bundle folder? This is a cache directory that includes, among other things, the Terraform-related binaries. By removing it, you'll force the CLI to re-initialize.

If the issue persists after doing this then something else must be going on...

[arielhgl]

I guess I should've mentioned this as well but yes I also tried removing the .databricks folder generated in the local repo bundle but the issue persists.

[github-actions[bot]]

This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.

[pietern]

@arielhgl If this still persists, could you check if you can execute the Terraform provider binary from the terminal?

If you open a terminal from VS Code, and you have tried performing deploy, you can find the provider binary at:

.databricks/bundle/dev/terraform/.terraform/providers/registry.terraform.io/databricks/databricks/1.65.1/darwin_arm64/terraform-provider-databricks_v1.65.1

The path may be slightly different for you.

If you execute this binary, you'll see some message saying it is a plugin and nothing else happens.

If you don't see that message, you might see the error that also happens during deployment.

I suspect what's happening is that Windows Defender rejects the execution of this binary.