terraform-provider-databricks
terraform-provider-databricks copied to clipboard
[ISSUE] Issue with `databricks_pipeline` resource - development = false does nothing
Configuration
resource "databricks_pipeline" "pipeline" {
provider = databricks.workspace
depends_on = [databricks_notebook.notebook_source]
name = var.pipeline_name
development = false
channel = "PREVIEW"
edition = "ADVANCED"
catalog = var.catalog_name
target = var.schema_name
configuration = {
env = var.environment
schema = var.schema_name
needAdminPermissionToViewLogs = false
}
cluster {
label = local.default_cluster.label
num_workers = local.default_cluster.num_workers
node_type_id = local.default_cluster.node_type_id
driver_node_type_id = local.default_cluster.driver_node_type_id
custom_tags = local.default_cluster.custom_tags
aws_attributes {
first_on_demand = 1
availability = "SPOT_WITH_FALLBACK"
spot_bid_price_percent = 75
}
}
Expected Behavior
When setting development = false in the terraform resource, a pipeline should be forced into production mode.
Actual Behavior
Setting development=false seems to have no effect on terraform state, it does not seem to compare the development mode with the current setting in the pipeline.
Steps to Reproduce
Terraform and provider versions
Terraform v1.9.2
on darwin_arm64
terraform {
required_providers {
databricks = {
source = "databricks/databricks"
version = "~> 1.48.2"
}
}
}
Is it a regression?
Debug Output
Important Factoids
I'm using this configuration in a local module to create a handful of DLT pipelines. I wouldn't think this would cause issues with the configuration not matching but wanted to mention it.
Would you like to implement a fix?
I'm not confident in my ability to look at/update the code for this.
Added more details to the issue template. I am not familiar with Go code but my assumption is there should be some sort of value for Development in the state information here? https://github.com/databricks/terraform-provider-databricks/blob/30107344b5d8a7678f8980e98cb5a60ce75fd12a/pipelines/resource_pipeline.go#L167-L176
Edit: well shoot I guess you kind of need the api to include the current value huh. https://docs.databricks.com/api/workspace/pipelines/get
I'll mention it to Databricks and see if they can add that to the GET API response.
EditEdit: nevermind, I does look like it's in there, under the spec. So seems doable?