Plan fails after rook-ceph cluster CRD is applied

[Isakgicu]

Terraform, Provider, Kubernetes versions

Terraform version: v0.13.5
Provider version: 0.2.1
Kubernetes version: v1.19.7

Affected Resource(s)

Terraform Configuration Files

provider.tf

provider "helm" {
  kubernetes {
    config_path = "/Users/isak/.kube/config"
  }
  version = "2.0.2"
}

terraform {
  required_providers {
    kubernetes-alpha = {
      source  = "hashicorp/kubernetes-alpha"
      version = "0.2.1"
    }
  }
}

provider "kubernetes-alpha" {
  config_path = "/Users/isak/.kube/config"
}

helm.tf

resource "helm_release" "rook-release" {
  name       = "local-storage-provisioner"
  repository = "https://charts.rook.io/release"
  chart      = "rook-ceph"
  namespace  = "rook-ceph"
}

ceph-cluster.tf

resource "kubernetes_manifest" "ceph-cluster" {
  provider = kubernetes-alpha
  manifest = {
    "apiVersion" = "ceph.rook.io/v1"
    "kind"       = "CephCluster"
    "metadata" = {
      "name"      = "rook-ceph"
      "namespace" = "rook-ceph"
    }
    "spec" = {
      "cephVersion" = {
        "image" = "ceph/ceph:v15.2.9"
      }
      "dataDirHostPath" = "/var/lib/rook"
      "mon" = {
        "allowMultiplePerNode" = false
        "count"                = 3
        "volumeClaimTemplate" = {
          "spec" = {
            "resources" = {
              "requests" = {
                "storage" = "10Gi"
              }
            }
            "storageClassName" = "local-path"
          }
        }
      }
      "storage" = {
        "storageClassDeviceSets" = [
          {
            "count"     = 16
            "encrypted" = false
            "name"      = "set1"
            "portable"  = false
            "volumeClaimTemplates" = [
              {
                "metadata" = {
                  "name" = "data"
                }
                "spec" = {
                  "accessModes" = [
                    "ReadWriteOnce",
                  ]
                  "resources" = {
                    "requests" = {
                      "storage" = "10Gi"
                    }
                  }
                  "storageClassName" = "local-storage"
                  "volumeMode"       = "Block"
                }
              },
            ]
          },
        ]
      }
    }
  }
}

Debug Output

https://gist.github.com/Isakgicu/e205ead448ac1f5a8212afe030bde642

Steps to Reproduce

1. In order to be able to apply rook CRDs in your kubernetes cluster, deploy rook-ceph helm chart with default values. (check helm.tf)
2. terraform apply for CRD resource provided above.
3. terraform plan after apply is completed.

Expected Behavior

terraform plan should return no changes to infrastructure

Actual Behavior

Error: rpc error: code = Unknown desc = update dry-run for 'rook-ceph/rook-ceph' failed: Apply failed with 1 conflict: conflict with "rook" using ceph.rook.io/v1: .spec.storage.storageClassDeviceSets

Community Note

Might be dublicate of: https://github.com/hashicorp/terraform-provider-kubernetes-alpha/issues/139

[Isakgicu]

Update:

Tried same thing with latest terraform-provider-kubernetes-alpha version: 0.3.1 and terraform 0.14.8. This time getting a different error:

https://gist.github.com/Isakgicu/3f3983c0d18840f6c6a7146dcceb907b