terraform-provider-databricks icon indicating copy to clipboard operation
terraform-provider-databricks copied to clipboard

[ISSUE] Issue with `databricks_pipeline` resource - development = false does nothing

Open adam133 opened this issue 1 year ago • 1 comments

Configuration

resource "databricks_pipeline" "pipeline" {
  provider   = databricks.workspace
  depends_on = [databricks_notebook.notebook_source]
  name       = var.pipeline_name
  development = false
  channel    = "PREVIEW"
  edition    = "ADVANCED"
  catalog    = var.catalog_name
  target     = var.schema_name
  configuration = {
    env    = var.environment
    schema = var.schema_name
    needAdminPermissionToViewLogs = false
  }
  cluster {
    label               = local.default_cluster.label
    num_workers         = local.default_cluster.num_workers
    node_type_id        = local.default_cluster.node_type_id
    driver_node_type_id = local.default_cluster.driver_node_type_id
    custom_tags         = local.default_cluster.custom_tags
    aws_attributes {
      first_on_demand        = 1
      availability           = "SPOT_WITH_FALLBACK"
      spot_bid_price_percent = 75
    }
  }

Expected Behavior

When setting development = false in the terraform resource, a pipeline should be forced into production mode.

Actual Behavior

Setting development=false seems to have no effect on terraform state, it does not seem to compare the development mode with the current setting in the pipeline.

Steps to Reproduce

Terraform and provider versions

Terraform v1.9.2
on darwin_arm64
terraform {
  required_providers {
    databricks = {
      source  = "databricks/databricks"
      version = "~> 1.48.2"
    }
  }
}

Is it a regression?

Debug Output

Important Factoids

I'm using this configuration in a local module to create a handful of DLT pipelines. I wouldn't think this would cause issues with the configuration not matching but wanted to mention it.

Would you like to implement a fix?

I'm not confident in my ability to look at/update the code for this.

adam133 avatar Jun 20 '24 16:06 adam133

Added more details to the issue template. I am not familiar with Go code but my assumption is there should be some sort of value for Development in the state information here? https://github.com/databricks/terraform-provider-databricks/blob/30107344b5d8a7678f8980e98cb5a60ce75fd12a/pipelines/resource_pipeline.go#L167-L176

Edit: well shoot I guess you kind of need the api to include the current value huh. https://docs.databricks.com/api/workspace/pipelines/get

I'll mention it to Databricks and see if they can add that to the GET API response.

EditEdit: nevermind, I does look like it's in there, under the spec. So seems doable?

adam133 avatar Jul 10 '24 17:07 adam133