Provider produced inconsistent result after apply

[ggolub]

Terraform, Provider, Kubernetes versions

Terraform version: v0.14.7
Provider version: 0.2.1/windows_amd64
Kubernetes version: Client Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.3", GitCommit:"1e11e4a2108024935ecfcb2912226cedeafd99df", GitTreeState:"clean", BuildDate:"2020-10-14T12:50:19Z", GoVersion:"go1.15.2", Compiler:"gc", Platform:"windows/amd64"}
Server Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.10", GitCommit:"41d24ec9c736cf0bdb0de3549d30c676e98eebaf", GitTreeState:"clean", BuildDate:"2021-01-18T09:12:27Z", GoVersion:"go1.13.15", Compiler:"gc", Platform:"linux/amd64"}

Affected Resource(s)

kubernetes_manifest

Terraform Configuration Files

# Copy-paste your Terraform configurations here - for large Terraform configs,
# please use a service like Dropbox and share a link to the ZIP file. For
# security, you can also encrypt the files using our GPG public key.

resource "kubernetes_manifest" "deployment_jaeger" { provider = kubernetes-alpha

manifest = { "apiVersion" = "apps/v1" "kind" = "Deployment" "metadata" = { "labels" = { "app" = "jaeger" } "name" = "jaeger" "namespace" = "istio-system" } "spec" = { "selector" = { "matchLabels" = { "app" = "jaeger" } } "template" = { "metadata" = { "annotations" = { "prometheus.io/port" = "14269" "prometheus.io/scrape" = "true" "sidecar.istio.io/inject" = "false" } "labels" = { "app" = "jaeger" } } "spec" = { "containers" = [ { "env" = [ { "name" = "BADGER_EPHEMERAL" "value" = "false" }, { "name" = "SPAN_STORAGE_TYPE" "value" = "badger" }, { "name" = "BADGER_DIRECTORY_VALUE" "value" = "/badger/data" }, { "name" = "BADGER_DIRECTORY_KEY" "value" = "/badger/key" }, { "name" = "MEMORY_MAX_TRACES" "value" = "50000" }, { "name" = "QUERY_BASE_PATH" "value" = "/jaeger" }, ] "image" = "docker.io/jaegertracing/all-in-one:1.20" "livenessProbe" = { "httpGet" = { "path" = "/" "port" = 14269 } } "name" = "jaeger" "readinessProbe" = { "httpGet" = { "path" = "/" "port" = 14269 } } "resources" = { "requests" = { "cpu" = "10m" } } "volumeMounts" = [ { "mountPath" = "/badger" "name" = "data" }, ] }, ] "volumes" = [ { "emptyDir" = {} "name" = "data" }, ] } } } } }

resource "kubernetes_manifest" "service_tracing" { provider = kubernetes-alpha

manifest = { "apiVersion" = "v1" "kind" = "Service" "metadata" = { "labels" = { "app" = "jaeger" } "name" = "tracing" "namespace" = "istio-system" } "spec" = { "ports" = [ { "name" = "http-query" "port" = 80 "protocol" = "TCP" "targetPort" = 16686 }, ] "selector" = { "app" = "jaeger" } "type" = "ClusterIP" } } }

resource "kubernetes_manifest" "service_jaeger_collector" { provider = kubernetes-alpha

manifest = { "apiVersion" = "v1" "kind" = "Service" "metadata" = { "labels" = { "app" = "jaeger" } "name" = "jaeger-collector" "namespace" = "istio-system" } "spec" = { "ports" = [ { "name" = "jaeger-collector-http" "port" = 14268 "protocol" = "TCP" "targetPort" = 14268 }, { "name" = "jaeger-collector-grpc" "port" = 14250 "protocol" = "TCP" "targetPort" = 14250 }, ] "selector" = { "app" = "jaeger" } "type" = "ClusterIP" } } }

Debug Output

Panic Output

Steps to Reproduce

terraform apply config.plan

Expected Behavior

What should have happened? Jaeger deployed to Kubernetes cluster

Actual Behavior

What actually happened? Got two errors after apply: Error: Provider produced inconsistent result after apply

When applying changes to kubernetes_manifest.service_jaeger_collector, provider "registry.terraform.io/hashicorp/kubernetes-alpha" produced an unexpected new value: .object: wrong final value type: incorrect object attributes.

This is a bug in the provider, which should be reported in the provider's own issue tracker.

Error: Provider produced inconsistent result after apply

When applying changes to kubernetes_manifest.service_tracing, provider "registry.terraform.io/hashicorp/kubernetes-alpha" produced an unexpected new value: .object: wrong final value type: incorrect object attributes.

This is a bug in the provider, which should be reported in the provider's own issue tracker.

Important Factoids

References

• https://github.com/hashicorp/terraform-provider-kubernetes-alpha/issues/76

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

[akojo]

I have the same issue when trying to create ManagedCertificates, using a definition like

resource "kubernetes_manifest" "acme-managedcert" {
  provider = kubernetes-alpha

  manifest = {
    "apiVersion" = "networking.gke.io/v1"
    "kind"       = "ManagedCertificate"
    "metadata" = {
      "name"      = "acme-certificate-${var.namespace}"
      "namespace" = var.namespace
    }
    "spec" = {
      "domains" = var.acme_domains
    }
  }
}

where acme_domains is a list(string) fails with:

│ Error: Provider produced inconsistent result after apply
│ 
│ When applying changes to
│ module.kubernetes.module.services.kubernetes_manifest.store-managedcert,
│ provider
│ "module.kubernetes.provider[\"registry.terraform.io/hashicorp/kubernetes-alpha\"]"
│ produced an unexpected new value: .object: wrong final value type:
│ incorrect object attributes.
│ 
│ This is a bug in the provider, which should be reported in the provider's
│ own issue tracker.

Using a hardcoded list of domains like

    "spec" = {
      "domains" = ["acme.example.com"]
    }

works without problems.

This makes me suspect that the plugin now doesn't convert lists and/or maps to correct HCL data types when reading them back from k8s.

[sambonbonne]

I have something similar with a namespace and its labels:

apiVersion: v1
kind: Namespace
metadata:
  labels:
    app.kubernetes.io/component: XXX
    app.kubernetes.io/name: YYY
    app.kubernetes.io/part-of: ZZZ
  name: test-namespace

Result when applying:

╷
│ Error: Provider produced inconsistent result after apply
│
│ When applying changes to module.test_kubernetes.kubernetes_manifest.test_namespace, provider
│ "provider[\"registry.terraform.io/hashicorp/kubernetes-alpha\"]" produced an unexpected new value: .object.metadata.labels: new element
│ "kubernetes.io/metadata.name" has appeared.
│
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.
╵

[gaurav517]

when applying:

resource "kubernetes_manifest" "istio-mci" {
  provider   = "kubernetes-alpha"
  manifest   = {
    apiVersion = "networking.gke.io/v1"
    kind       = "MultiClusterIngress"
    metadata   = {
      annotations = {
        "networking.gke.io/pre-shared-certs" = "my-cert"
        "networking.gke.io/static-ip"        = "1.2.3.4"
      }
      "name"      = "istio-multicluster-ingress"
      "namespace" = "istio-system"
    }
    "spec"     = {
      "template" = {
        "spec" = {
          "backend" = {
            "serviceName" = "mcs-istio-ingress"
            "servicePort" = 443
          }
        }
      }
    }
  }
}

Get error:

╷
│ Error: Provider produced inconsistent result after apply
│ 
│ When applying changes to kubernetes_manifest.istio-mci, provider "provider[\"registry.terraform.io/hashicorp/kubernetes-alpha\"]" produced
│ an unexpected new value: .object.metadata.annotations: new element "networking.gke.io/last-reconcile-time" has appeared.
│ 
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.
╵

Is there any workaround for this? Thanks.