terraform-provider-kubernetes icon indicating copy to clipboard operation
terraform-provider-kubernetes copied to clipboard

Published 20 hours ago • verified publisher icon hashicorp

Plugin Crash with Error: The terraform-provider-kubernetes_v2.27.0_x5 plugin crashed

Open GaneshKudikala opened this issue 1 year ago • 9 comments

The terraform-provider-kubernetes plugin crashed with the following error message:

Error: The terraform-provider-kubernetes_v2.27.0_x5 plugin crashed!

This is always indicative of a bug within the plugin. It would be immensely helpful if you could report the crash with the plugin's maintainers so that it can be fixed. The output above should help diagnose the issue.

Terraform Version, Provider Version and Kubernetes Version

Terraform version: Terraform v1.6.4
Kubernetes provider version: kubernetes v2.27.0
Kubernetes version: [Client Version: v1.28.3
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.29.0-eks-c417bb3]

Affected Resource(s)

Terraform Configuration Files

# Copy-paste your Terraform configurations here - for large Terraform configs,
# please use a service like Dropbox and share a link to the ZIP file. For
# security, you can also encrypt the files using our GPG public key.


data "kubernetes_resources" "deployments" {
   for_each    = toset(data.kubernetes_all_namespaces.allns.namespaces)
    api_version = "apps/v1"
  # Query all deployments
  kind = "Deployment"

  # Optionally, filter deployments by namespace
  namespace = each.key
}

provider "kubernetes" {
    config_path    = "~/.kube/config"
}

data "kubernetes_all_namespaces" "allns" {
}

Here is the Terraform File, The issue occors only if the Deployment/apps have 3 or more deployments, So if you want to recreate the error, Deploy EKS cluster with each namespace with 3+ depployments are may be you can see the error.

Debug Output

https://gist.github.com/GaneshKudikala/3a97545ec3cb4776c55754a4d957314d

Panic Output

Steps to Reproduce

Expected Behavior

What should have happened?

Actual Behavior

What actually happened?

Important Factoids

References

  • GH-1234 https://github.com/hashicorp/terraform-provider-kubernetes/issues/2430

Community Note

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment

GaneshKudikala avatar Mar 07 '24 14:03 GaneshKudikala

@GaneshKudikala Hi thanks for opening this issue. Can you provide us with the YAML that you've used for your deployments? It will help with reproducing this.

BBBmau avatar Mar 07 '24 21:03 BBBmau

To resproduce the same error I have made the following steps:

  1. Deploy a Kind Kuberntes cluster with following configuration
# three node (two workers) cluster config
  kind: Cluster
  apiVersion: kind.x-k8s.io/v1alpha4
  nodes:
  - role: control-plane
  - role: worker
  - role: worker

  1. Deploy sample applications from the repo https://github.com/digitalocean/kubernetes-sample-apps kubectl apply -k bookinfo-example/kustomize kubectl apply -k emojivoto-example/kustomize kubectl apply -k podinfo-example/kustomize

  2. The following terraform code was used to retrive the resources from the cluster for each namespaces. `

data "kubernetes_resources" "deployments" {
  for_each = toset(data.kubernetes_all_namespaces.allns.namespaces)
  api_version = "apps/v1"
  kind        = "Deployment"
  namespace   = each.key
}

provider "kubernetes" {
 config_path    = "~/.kube/config"
  config_context = "kind-k8s-playground"

}


data "kubernetes_all_namespaces" "allns" {}`
  1. Try terraform init and terraform plan to give us the Error
`goroutine 67 [running]:
github.com/hashicorp/terraform-plugin-go/tftypes.NewValue(...)
	github.com/hashicorp/[email protected]/tftypes/value.go:278
github.com/hashicorp/terraform-provider-kubernetes/manifest/morph.UnknownToNull({{0x24c8ec0?, 0xc00a27f1d0?}, {0x1e62300?, 0xc00a25a5d0?}})
	github.com/hashicorp/terraform-provider-kubernetes/manifest/morph/scaffold.go:143 +0x508
github.com/hashicorp/terraform-provider-kubernetes/manifest/morph.UnknownToNull({{0x24c8ec0?, 0xc00a305c50?}, {0x1e62300?, 0xc00a25a540?}})
	github.com/hashicorp/terraform-provider-kubernetes/manifest/morph/scaffold.go:141 +0x438
github.com/hashicorp/terraform-provider-kubernetes/manifest/morph.UnknownToNull({{0x24c8ec0?, 0xc00a30b710?}, {0x1e62300?, 0xc00a259710?}})
	github.com/hashicorp/terraform-provider-kubernetes/manifest/morph/scaffold.go:141 +0x438
github.com/hashicorp/terraform-provider-kubernetes/manifest/morph.UnknownToNull({{0x24c8ec0?, 0xc00a3120c0?}, {0x1e62300?, 0xc00a243770?}})
	github.com/hashicorp/terraform-provider-kubernetes/manifest/morph/scaffold.go:141 +0x438
github.com/hashicorp/terraform-provider-kubernetes/manifest/morph.UnknownToNull({{0x24c9128?, 0xc007b5f020?}, {0x1d393a0?, 0xc00470a1f8?}})
	github.com/hashicorp/terraform-provider-kubernetes/manifest/morph/scaffold.go:134 +0x566
github.com/hashicorp/terraform-provider-kubernetes/manifest/provider.(*RawProviderServer).ReadPluralDataSource(0xc00019ca80, {0x24c1f48, 0xc000f21f20}, 0xc0004b9480)
	github.com/hashicorp/terraform-provider-kubernetes/manifest/provider/datasource.go:218 +0x1c11
github.com/hashicorp/terraform-provider-kubernetes/manifest/provider.(*RawProviderServer).ReadDataSource(0x24c1f48?, {0x24c1f48, 0xc000f21f20}, 0xc0004b9480)
	github.com/hashicorp/terraform-provider-kubernetes/manifest/provider/datasource.go:28 +0x85
github.com/hashicorp/terraform-plugin-mux/tf5muxserver.(*muxServer).ReadDataSource(0x24c1f80?, {0x24c1f48?, 0xc000f21c20?}, 0xc0004b9480)
	github.com/hashicorp/[email protected]/tf5muxserver/mux_server_ReadDataSource.go:36 +0x19f
github.com/hashicorp/terraform-plugin-go/tfprotov5/tf5server.(*server).ReadDataSource(0xc000468500, {0x24c1f48?, 0xc000f21470?}, 0xc0004d2190)
	github.com/hashicorp/[email protected]/tfprotov5/tf5server/server.go:699 +0x3df
github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/tfplugin5._Provider_ReadDataSource_Handler({0x20c32c0?, 0xc000468500}, {0x24c1f48, 0xc000f21470}, 0xc0002b5380, 0x0)
	github.com/hashicorp/[email protected]/tfprotov5/internal/tfplugin5/tfplugin5_grpc.pb.go:539 +0x169
google.golang.org/grpc.(*Server).processUnaryRPC(0xc000b7e000, {0x24c1f48, 0xc000f213e0}, {0x24ca2a0, 0xc00074b6c0}, 0xc000f27e60, 0xc000b822a0, 0x3546dc8, 0x0)
	google.golang.org/[email protected]/server.go:1372 +0xe03
google.golang.org/grpc.(*Server).handleStream(0xc000b7e000, {0x24ca2a0, 0xc00074b6c0}, 0xc000f27e60)
	google.golang.org/[email protected]/server.go:1783 +0xfec
google.golang.org/grpc.(*Server).serveStreams.func2.1()
	google.golang.org/[email protected]/server.go:1016 +0x59
created by google.golang.org/grpc.(*Server).serveStreams.func2 in goroutine 25
	google.golang.org/[email protected]/server.go:1027 +0x115

Error: The terraform-provider-kubernetes_v2.27.0_x5 plugin crashed!

This is always indicative of a bug within the plugin. It would be immensely helpful if you could report the crash with the plugin's maintainers so that it can be fixed. The output above should help diagnose the issue. `

GaneshKudikala avatar Mar 13 '24 16:03 GaneshKudikala

This issue is not version specific, we had the same behavior with previous versions of the kubernetes provider.

xego avatar Mar 14 '24 08:03 xego

It looks like manifects/data sources - kubernetes_resources are broken. it failed for me with latest version of provider (v.2.29.0) with example from documentation (getting all namespaces via manifest/kubernetes_resources datasource): https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/data-sources/resources#example-get-a-list-of-namespaces-excluding-kube-system-using-field_selector I aslo can confim from TF_TRACE that data was fetched from k8s and it fails in github.com/hashicorp/terraform-provider-kubernetes/manifest/morph.UnknownToNull I guess it's the same issue and there's no reason to create another one.

Also, this can be related/same for https://github.com/hashicorp/terraform-provider-kubernetes/issues/2215

PavelSlepushkin avatar Apr 12 '24 11:04 PavelSlepushkin

Thank you for the extensive instructions, @GaneshKudikala I was able to reproduce following your steps. It seems we already had an open PR that addresses this issue and it somehow slipped through the cracks and never got merged.

The fix for this will be available in the next provider release.

To resproduce the same error I have made the following steps:

...

alexsomesan avatar Apr 17 '24 18:04 alexsomesan

@alexsomesan I have seen the new release version v2.30.0, i have upgraded my plugin to use the new version however this new version may be doesn't have the fix(or Correct me if i am wrong), Still crashing

GaneshKudikala avatar May 15 '24 07:05 GaneshKudikala

Got same problem with version terraform-provider-kubernetes_v2.31.0_x5

SalehBorhani avatar Jun 19 '24 09:06 SalehBorhani

Same crash for me. It works on 2.29.0, but on 2.31.0 crashes with stacktrace below. On test environment configuration was applied on the first run and I compared terraform states. What I found is that lists were converted to nested tuples, please, see the screenshot attached. image image

Stack trace from the terraform-provider-kubernetes_v2.31.0_x5 plugin:

panic: ElementKeyInt(0): can't use tftypes.Object["attach_metadata":tftypes.Object["node":tftypes.Bool], "authorization":tftypes.Object["credentials":tftypes.Object["key":tftypes.String, "name":tftypes.String, "optional":tftypes.Bool], "credentialsFile":tftypes.String, "type":tftypes.String], "basicAuth":tftypes.Object["password":tftypes.Object["key":tftypes.String, "name":tftypes.String, "optional":tftypes.Bool], "password_file":tftypes.String, "username":tftypes.Object["key":tftypes.String, "name":tftypes.String, "optional":tftypes.Bool]], "bearerTokenFile":tftypes.String, "bearerTokenSecret":tftypes.Object["key":tftypes.String, "name":tftypes.String, "optional":tftypes.Bool], "follow_redirects":tftypes.Bool, "honorLabels":tftypes.Bool, "honorTimestamps":tftypes.Bool, "interval":tftypes.String, "metricRelabelConfigs":tftypes.Tuple[tftypes.Object["action":tftypes.String, "if":tftypes.DynamicPseudoType, "labels":tftypes.Map[tftypes.String], "match":tftypes.String, "modulus":tftypes.Number, "regex":tftypes.DynamicPseudoType, "replacement":tftypes.String, "separator":tftypes.String, "sourceLabels":tftypes.List[tftypes.String], "source_labels":tftypes.List[tftypes.String], "targetLabel":tftypes.String, "target_label":tftypes.String], tftypes.Object["action":tftypes.String, "if":tftypes.DynamicPseudoType, "labels":tftypes.Map[tftypes.String], "match":tftypes.String, "modulus":tftypes.Number, "regex":tftypes.DynamicPseudoType, "replacement":tftypes.String, "separator":tftypes.String, "sourceLabels":tftypes.List[tftypes.String], "source_labels":tftypes.List[tftypes.String], "targetLabel":tftypes.String, "target_label":tftypes.String]], "oauth2":tftypes.Object["client_id":tftypes.Object["configMap":tftypes.Object["key":tftypes.String, "name":tftypes.String, "optional":tftypes.Bool], "secret":tftypes.Object["key":tftypes.String, "name":tftypes.String, "optional":tftypes.Bool]], "client_secret":tftypes.Object["key":tftypes.String, "name":tftypes.String, "optional":tftypes.Bool], "client_secret_file":tftypes.String, "endpoint_params":tftypes.Map[tftypes.String], "scopes":tftypes.List[tftypes.String], "token_url":tftypes.String], "params":tftypes.Map[tftypes.List[tftypes.String]], "path":tftypes.String, "port":tftypes.String, "proxyURL":tftypes.String, "relabelConfigs":tftypes.Tuple[tftypes.Object["action":tftypes.String, "if":tftypes.DynamicPseudoType, "labels":tftypes.Map[tftypes.String], "match":tftypes.String, "modulus":tftypes.Number, "regex":tftypes.DynamicPseudoType, "replacement":tftypes.String, "separator":tftypes.String, "sourceLabels":tftypes.List[tftypes.String], "source_labels":tftypes.List[tftypes.String], "targetLabel":tftypes.String, "target_label":tftypes.String]], "sampleLimit":tftypes.Number, "scheme":tftypes.String, "scrapeTimeout":tftypes.String, "scrape_interval":tftypes.String, "seriesLimit":tftypes.Number, "targetPort":tftypes.String, "tlsConfig":tftypes.Object["ca":tftypes.Object["configMap":tftypes.Object["key":tftypes.String, "name":tftypes.String, "optional":tftypes.Bool], "secret":tftypes.Object["key":tftypes.String, "name":tftypes.String, "optional":tftypes.Bool]], "caFile":tftypes.String, "cert":tftypes.Object["configMap":tftypes.Object["key":tftypes.String, "name":tftypes.String, "optional":tftypes.Bool], "secret":tftypes.Object["key":tftypes.String, "name":tftypes.String, "optional":tftypes.Bool]], "certFile":tftypes.String, "insecureSkipVerify":tftypes.Bool, "keyFile":tftypes.String, "keySecret":tftypes.Object["key":tftypes.String, "name":tftypes.String, "optional":tftypes.Bool], "serverName":tftypes.String], "vm_scrape_params":tftypes.Object["disable_compression":tftypes.Bool, "disable_keep_alive":tftypes.Bool, "headers":tftypes.List[tftypes.String], "metric_relabel_debug":tftypes.Bool, "no_stale_markers":tftypes.Bool, "proxy_client_config":tftypes.Object["basic_auth":tftypes.Object["password":tftypes.Object["key":tftypes.String, "name":tftypes.String, "optional":tftypes.Bool], "password_file":tftypes.String, "username":tftypes.Object["key":tftypes.String, "name":tftypes.String, "optional":tftypes.Bool]], "bearer_token":tftypes.Object["key":tftypes.String, "name":tftypes.String, "optional":tftypes.Bool], "bearer_token_file":tftypes.String, "tls_config":tftypes.Object["ca":tftypes.Object["configMap":tftypes.Object["key":tftypes.String, "name":tftypes.String, "optional":tftypes.Bool], "secret":tftypes.Object["key":tftypes.String, "name":tftypes.String, "optional":tftypes.Bool]], "caFile":tftypes.String, "cert":tftypes.Object["configMap":tftypes.Object["key":tftypes.String, "name":tftypes.String, "optional":tftypes.Bool], "secret":tftypes.Object["key":tftypes.String, "name":tftypes.String, "optional":tftypes.Bool]], "certFile":tftypes.String, "insecureSkipVerify":tftypes.Bool, "keyFile":tftypes.String, "keySecret":tftypes.Object["key":tftypes.String, "name":tftypes.String, "optional":tftypes.Bool], "serverName":tftypes.String]], "relabel_debug":tftypes.Bool, "scrape_align_interval":tftypes.String, "scrape_offset":tftypes.String, "stream_parse":tftypes.Bool]] as tftypes.Object["attach_metadata":tftypes.Object["node":tftypes.Bool], "authorization":tftypes.Object["credentials":tftypes.Object["key":tftypes.String, "name":tftypes.String, "optional":tftypes.Bool], "credentialsFile":tftypes.String, "type":tftypes.String], "basicAuth":tftypes.Object["password":tftypes.Object["key":tftypes.String, "name":tftypes.String, "optional":tftypes.Bool], "password_file":tftypes.String, "username":tftypes.Object["key":tftypes.String, "name":tftypes.String, "optional":tftypes.Bool]], "bearerTokenFile":tftypes.String, "bearerTokenSecret":tftypes.Object["key":tftypes.String, "name":tftypes.String, "optional":tftypes.Bool], "follow_redirects":tftypes.Bool, "honorLabels":tftypes.Bool, "honorTimestamps":tftypes.Bool, "interval":tftypes.String, "metricRelabelConfigs":tftypes.Tuple[tftypes.Object["action":tftypes.String, "if":tftypes.DynamicPseudoType, "labels":tftypes.Map[tftypes.String], "match":tftypes.String, "modulus":tftypes.Number, "regex":tftypes.DynamicPseudoType, "replacement":tftypes.String, "separator":tftypes.String, "sourceLabels":tftypes.List[tftypes.String], "source_labels":tftypes.List[tftypes.String], "targetLabel":tftypes.String, "target_label":tftypes.String]], "oauth2":tftypes.Object["client_id":tftypes.Object["configMap":tftypes.Object["key":tftypes.String, "name":tftypes.String, "optional":tftypes.Bool], "secret":tftypes.Object["key":tftypes.String, "name":tftypes.String, "optional":tftypes.Bool]], "client_secret":tftypes.Object["key":tftypes.String, "name":tftypes.String, "optional":tftypes.Bool], "client_secret_file":tftypes.String, "endpoint_params":tftypes.Map[tftypes.String], "scopes":tftypes.List[tftypes.String], "token_url":tftypes.String], "params":tftypes.Map[tftypes.List[tftypes.String]], "path":tftypes.String, "port":tftypes.String, "proxyURL":tftypes.String, "relabelConfigs":tftypes.Tuple[tftypes.Object["action":tftypes.String, "if":tftypes.DynamicPseudoType, "labels":tftypes.Map[tftypes.String], "match":tftypes.String, "modulus":tftypes.Number, "regex":tftypes.DynamicPseudoType, "replacement":tftypes.String, "separator":tftypes.String, "sourceLabels":tftypes.List[tftypes.String], "source_labels":tftypes.List[tftypes.String], "targetLabel":tftypes.String, "target_label":tftypes.String]], "sampleLimit":tftypes.Number, "scheme":tftypes.String, "scrapeTimeout":tftypes.String, "scrape_interval":tftypes.String, "seriesLimit":tftypes.Number, "targetPort":tftypes.String, "tlsConfig":tftypes.Object["ca":tftypes.Object["configMap":tftypes.Object["key":tftypes.String, "name":tftypes.String, "optional":tftypes.Bool], "secret":tftypes.Object["key":tftypes.String, "name":tftypes.String, "optional":tftypes.Bool]], "caFile":tftypes.String, "cert":tftypes.Object["configMap":tftypes.Object["key":tftypes.String, "name":tftypes.String, "optional":tftypes.Bool], "secret":tftypes.Object["key":tftypes.String, "name":tftypes.String, "optional":tftypes.Bool]], "certFile":tftypes.String, "insecureSkipVerify":tftypes.Bool, "keyFile":tftypes.String, "keySecret":tftypes.Object["key":tftypes.String, "name":tftypes.String, "optional":tftypes.Bool], "serverName":tftypes.String], "vm_scrape_params":tftypes.Object["disable_compression":tftypes.Bool, "disable_keep_alive":tftypes.Bool, "headers":tftypes.List[tftypes.String], "metric_relabel_debug":tftypes.Bool, "no_stale_markers":tftypes.Bool, "proxy_client_config":tftypes.Object["basic_auth":tftypes.Object["password":tftypes.Object["key":tftypes.String, "name":tftypes.String, "optional":tftypes.Bool], "password_file":tftypes.String, "username":tftypes.Object["key":tftypes.String, "name":tftypes.String, "optional":tftypes.Bool]], "bearer_token":tftypes.Object["key":tftypes.String, "name":tftypes.String, "optional":tftypes.Bool], "bearer_token_file":tftypes.String, "tls_config":tftypes.Object["ca":tftypes.Object["configMap":tftypes.Object["key":tftypes.String, "name":tftypes.String, "optional":tftypes.Bool], "secret":tftypes.Object["key":tftypes.String, "name":tftypes.String, "optional":tftypes.Bool]], "caFile":tftypes.String, "cert":tftypes.Object["configMap":tftypes.Object["key":tftypes.String, "name":tftypes.String, "optional":tftypes.Bool], "secret":tftypes.Object["key":tftypes.String, "name":tftypes.String, "optional":tftypes.Bool]], "certFile":tftypes.String, "insecureSkipVerify":tftypes.Bool, "keyFile":tftypes.String, "keySecret":tftypes.Object["key":tftypes.String, "name":tftypes.String, "optional":tftypes.Bool], "serverName":tftypes.String]], "relabel_debug":tftypes.Bool, "scrape_align_interval":tftypes.String, "scrape_offset":tftypes.String, "stream_parse":tftypes.Bool]]

goroutine 93 [running]:
github.com/hashicorp/terraform-plugin-go/tftypes.NewValue(...)
	github.com/hashicorp/[email protected]/tftypes/value.go:278
github.com/hashicorp/terraform-provider-kubernetes/manifest/morph.DeepUnknown({0x3129a18, 0xc00442e120?}, {{0x3129a18?, 0xc0044b33b0?}, {0x2980a60?, 0xc00dd9ac30?}}, 0xc00dd9b350)
	github.com/hashicorp/terraform-provider-kubernetes/manifest/morph/scaffold.go:86 +0x19ae
github.com/hashicorp/terraform-provider-kubernetes/manifest/morph.DeepUnknown({0x3129580, 0xc004436870?}, {{0x3129580?, 0xc0044b4d50?}, {0x2aae300?, 0xc0044a5230?}}, 0xc00dd9b0f8)
	github.com/hashicorp/terraform-provider-kubernetes/manifest/morph/scaffold.go:33 +0x1cb4
github.com/hashicorp/terraform-provider-kubernetes/manifest/morph.DeepUnknown({0x3129580, 0xc0044a4db0?}, {{0x3129580?, 0xc0044c61b0?}, {0x2aae300?, 0xc0044a5020?}}, 0xc00dd9aed0)
	github.com/hashicorp/terraform-provider-kubernetes/manifest/morph/scaffold.go:33 +0x1cb4
github.com/hashicorp/terraform-provider-kubernetes/manifest/provider.(*RawProviderServer).PlanResourceChange(0xc0001d1500, {0x3122558, 0xc0011ba1e0}, 0xc0009b75e0)
	github.com/hashicorp/terraform-provider-kubernetes/manifest/provider/plan.go:388 +0x3273
github.com/hashicorp/terraform-plugin-mux/tf5muxserver.(*muxServer).PlanResourceChange(0xc00021ae00, {0x3122558?, 0xc0011b3ec0?}, 0xc0009b75e0)
	github.com/hashicorp/[email protected]/tf5muxserver/mux_server_PlanResourceChange.go:73 +0x2ad
github.com/hashicorp/terraform-plugin-go/tfprotov5/tf5server.(*server).PlanResourceChange(0xc000186640, {0x3122558?, 0xc0011b3620?}, 0xc000333780)
	github.com/hashicorp/[email protected]/tfprotov5/tf5server/server.go:825 +0x3f0
github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/tfplugin5._Provider_PlanResourceChange_Handler({0x2d1e4e0?, 0xc000186640}, {0x3122558, 0xc0011b3620}, 0xc000333700, 0x0)
	github.com/hashicorp/[email protected]/tfprotov5/internal/tfplugin5/tfplugin5_grpc.pb.go:500 +0x169
google.golang.org/grpc.(*Server).processUnaryRPC(0xc0001cce00, {0x3122558, 0xc0011b3590}, {0x312a9e8, 0xc000215500}, 0xc00116a000, 0xc000c20f30, 0x41d3b40, 0x0)
	google.golang.org/[email protected]/server.go:1369 +0xe23
google.golang.org/grpc.(*Server).handleStream(0xc0001cce00, {0x312a9e8, 0xc000215500}, 0xc00116a000)
	google.golang.org/[email protected]/server.go:1780 +0x1016
google.golang.org/grpc.(*Server).serveStreams.func2.1()
	google.golang.org/[email protected]/server.go:1019 +0x8b
created by google.golang.org/grpc.(*Server).serveStreams.func2 in goroutine 55
	google.golang.org/[email protected]/server.go:1030 +0x135

Error: The terraform-provider-kubernetes_v2.31.0_x5 plugin crashed!

This is always indicative of a bug within the plugin. It would be immensely
helpful if you could report the crash with the plugin's maintainers so that it
can be fixed. The output above should help diagnose the issue.

rkul avatar Jul 22 '24 23:07 rkul

Team, is there any update on this? If a Fix is not provided soon we will have to avoid using this plugin at all.

Stack trace from the terraform-provider-kubernetes_v2.32.0_x5 plugin:

panic: ElementKeyInt(0): can't use tftypes.Object["deployStartedAt":tftypes.String, "deployedAt":tftypes.String, "id":tftypes.Number, "revision":tftypes.String, "revisions":tftypes.List[tftypes.String], "source":tftypes.Object["chart":tftypes.String, "directory":tftypes.Object["exclude":tftypes.String, "include":tftypes.String, "jsonnet":tftypes.Object["extVars":tftypes.List[tftypes.Object["code":tftypes.Bool, "name":tftypes.String, "value":tftypes.String]], "libs":tftypes.List[tftypes.String], "tlas":tftypes.List[tftypes.Object["code":tftypes.Bool, "name":tftypes.String, "value":tftypes.String]]], "recurse":tftypes.Bool], "helm":tftypes.Object["fileParameters":tftypes.List[tftypes.Object["name":tftypes.String, "path":tftypes.String]], "ignoreMissingValueFiles":tftypes.Bool, "parameters":tftypes.List[tftypes.Object["forceString":tftypes.Bool, "name":tftypes.String, "value":tftypes.String]], "passCredentials":tftypes.Bool, "releaseName":tftypes.String, "skipCrds":tftypes.Bool, "valueFiles":tftypes.List[tftypes.String], "values":tftypes.String, "valuesObject":tftypes.DynamicPseudoType, "version":tftypes.String], "kustomize":tftypes.Object["commonAnnotations":tftypes.Map[tftypes.String], "commonAnnotationsEnvsubst":tftypes.Bool, "commonLabels":tftypes.Map[tftypes.String], "components":tftypes.List[tftypes.String], "forceCommonAnnotations":tftypes.Bool, "forceCommonLabels":tftypes.Bool, "images":tftypes.List[tftypes.String], "namePrefix":tftypes.String, "nameSuffix":tftypes.String, "namespace":tftypes.String, "patches":tftypes.List[tftypes.Object["options":tftypes.Map[tftypes.Bool], "patch":tftypes.String, "path":tftypes.String, "target":tftypes.Object["annotationSelector":tftypes.String, "group":tftypes.String, "kind":tftypes.String, "labelSelector":tftypes.String, "name":tftypes.String, "namespace":tftypes.String, "version":tftypes.String]]], "replicas":tftypes.List[tftypes.Object["count":tftypes.String, "name":tftypes.String]], "version":tftypes.String], "path":tftypes.String, "plugin":tftypes.Object["env":tftypes.List[tftypes.Object["name":tftypes.String, "value":tftypes.String]], "name":tftypes.String, "parameters":tftypes.List[tftypes.Object["array":tftypes.List[tftypes.String], "map":tftypes.Map[tftypes.String], "name":tftypes.String, "string":tftypes.String]]], "ref":tftypes.String, "repoURL":tftypes.String, "targetRevision":tftypes.String], "sources":tftypes.Tuple[tftypes.Object["chart":tftypes.String, "directory":tftypes.Object["exclude":tftypes.String, "include":tftypes.String, "jsonnet":tftypes.Object["extVars":tftypes.List[tftypes.Object["code":tftypes.Bool, "name":tftypes.String, "value":tftypes.String]], "libs":tftypes.List[tftypes.String], "tlas":tftypes.List[tftypes.Object["code":tftypes.Bool, "name":tftypes.String, "value":tftypes.String]]], "recurse":tftypes.Bool], "helm":tftypes.Object["fileParameters":tftypes.List[tftypes.Object["name":tftypes.String, "path":tftypes.String]], "ignoreMissingValueFiles":tftypes.Bool, "parameters":tftypes.List[tftypes.Object["forceString":tftypes.Bool, "name":tftypes.String, "value":tftypes.String]], "passCredentials":tftypes.Bool, "releaseName":tftypes.String, "skipCrds":tftypes.Bool, "valueFiles":tftypes.List[tftypes.String], "values":tftypes.String, "valuesObject":tftypes.DynamicPseudoType, "version":tftypes.String], "kustomize":tftypes.Object["commonAnnotations":tftypes.Map[tftypes.String], "commonAnnotationsEnvsubst":tftypes.Bool, "commonLabels":tftypes.Map[tftypes.String], "components":tftypes.List[tftypes.String], "forceCommonAnnotations":tftypes.Bool, "forceCommonLabels":tftypes.Bool, "images":tftypes.List[tftypes.String], "namePrefix":tftypes.String, "nameSuffix":tftypes.String, "namespace":tftypes.String, "patches":tftypes.List[tftypes.Object["options":tftypes.Map[tftypes.Bool], "patch":tftypes.String, "path":tftypes.String, "target":tftypes.Object["annotationSelector":tftypes.String, "group":tftypes.String, "kind":tftypes.String, "labelSelector":tftypes.String, "name":tftypes.String, "namespace":tftypes.String, "version":tftypes.String]]], "replicas":tftypes.List[tftypes.Object["count":tftypes.String, "name":tftypes.String]], "version":tftypes.String], "path":tftypes.String, "plugin":tftypes.Object["env":tftypes.List[tftypes.Object["name":tftypes.String, "value":tftypes.String]], "name":tftypes.String, "parameters":tftypes.List[tftypes.Object["array":tftypes.List[tftypes.String], "map":tftypes.Map[tftypes.String], "name":tftypes.String, "string":tftypes.String]]], "ref":tftypes.String, "repoURL":tftypes.String, "targetRevision":tftypes.String], tftypes.Object["chart":tftypes.String, "directory":tftypes.Object["exclude":tftypes.String, "include":tftypes.String, "jsonnet":tftypes.Object["extVars":tftypes.List[tftypes.Object["code":tftypes.Bool, "name":tftypes.String, "value":tftypes.String]], "libs":tftypes.List[tftypes.String], "tlas":tftypes.List[tftypes.Object["code":tftypes.Bool, "name":tftypes.String, "value":tftypes.String]]], "recurse":tftypes.Bool], "helm":tftypes.Object["fileParameters":tftypes.List[tftypes.Object["name":tftypes.String, "path":tftypes.String]], "ignoreMissingValueFiles":tftypes.Bool, "parameters":tftypes.List[tftypes.Object["forceString":tftypes.Bool, "name":tftypes.String, "value":tftypes.String]], "passCredentials":tftypes.Bool, "releaseName":tftypes.String, "skipCrds":tftypes.Bool, "valueFiles":tftypes.List[tftypes.String], "values":tftypes.String, "valuesObject":tftypes.DynamicPseudoType, "version":tftypes.String], "kustomize":tftypes.Object["commonAnnotations":tftypes.Map[tftypes.String], "commonAnnotationsEnvsubst":tftypes.Bool, "commonLabels":tftypes.Map[tftypes.String], "components":tftypes.List[tftypes.String], "forceCommonAnnotations":tftypes.Bool, "forceCommonLabels":tftypes.Bool, "images":tftypes.List[tftypes.String], "namePrefix":tftypes.String, "nameSuffix":tftypes.String, "namespace":tftypes.String, "patches":tftypes.List[tftypes.Object["options":tftypes.Map[tftypes.Bool], "patch":tftypes.String, "path":tftypes.String, "target":tftypes.Object["annotationSelector":tftypes.String, "group":tftypes.String, "kind":tftypes.String, "labelSelector":tftypes.String, "name":tftypes.String, "namespace":tftypes.String, "version":tftypes.String]]], "replicas":tftypes.List[tftypes.Object["count":tftypes.String, "name":tftypes.String]], "version":tftypes.String], "path":tftypes.String, "plugin":tftypes.Object["env":tftypes.List[tftypes.Object["name":tftypes.String, "value":tftypes.String]], "name":tftypes.String, "parameters":tftypes.List[tftypes.Object["array":tftypes.List[tftypes.String], "map":tftypes.Map[tftypes.String], "name":tftypes.String, "string":tftypes.String]]], "ref":tftypes.String, "repoURL":tftypes.String, "targetRevision":tftypes.String]]] as tftypes.Object["deployStartedAt":tftypes.String, "deployedAt":tftypes.String, "id":tftypes.Number, "revision":tftypes.String, "revisions":tftypes.List[tftypes.String], "source":tftypes.Object["chart":tftypes.String, "directory":tftypes.Object["exclude":tftypes.String, "include":tftypes.String, "jsonnet":tftypes.Object["extVars":tftypes.List[tftypes.Object["code":tftypes.Bool, "name":tftypes.String, "value":tftypes.String]], "libs":tftypes.List[tftypes.String], "tlas":tftypes.List[tftypes.Object["code":tftypes.Bool, "name":tftypes.String, "value":tftypes.String]]], "recurse":tftypes.Bool], "helm":tftypes.Object["fileParameters":tftypes.List[tftypes.Object["name":tftypes.String, "path":tftypes.String]], "ignoreMissingValueFiles":tftypes.Bool, "parameters":tftypes.List[tftypes.Object["forceString":tftypes.Bool, "name":tftypes.String, "value":tftypes.String]], "passCredentials":tftypes.Bool, "releaseName":tftypes.String, "skipCrds":tftypes.Bool, "valueFiles":tftypes.List[tftypes.String], "values":tftypes.String, "valuesObject":tftypes.DynamicPseudoType, "version":tftypes.String], "kustomize":tftypes.Object["commonAnnotations":tftypes.Map[tftypes.String], "commonAnnotationsEnvsubst":tftypes.Bool, "commonLabels":tftypes.Map[tftypes.String], "components":tftypes.List[tftypes.String], "forceCommonAnnotations":tftypes.Bool, "forceCommonLabels":tftypes.Bool, "images":tftypes.List[tftypes.String], "namePrefix":tftypes.String, "nameSuffix":tftypes.String, "namespace":tftypes.String, "patches":tftypes.List[tftypes.Object["options":tftypes.Map[tftypes.Bool], "patch":tftypes.String, "path":tftypes.String, "target":tftypes.Object["annotationSelector":tftypes.String, "group":tftypes.String, "kind":tftypes.String, "labelSelector":tftypes.String, "name":tftypes.String, "namespace":tftypes.String, "version":tftypes.String]]], "replicas":tftypes.List[tftypes.Object["count":tftypes.String, "name":tftypes.String]], "version":tftypes.String], "path":tftypes.String, "plugin":tftypes.Object["env":tftypes.List[tftypes.Object["name":tftypes.String, "value":tftypes.String]], "name":tftypes.String, "parameters":tftypes.List[tftypes.Object["array":tftypes.List[tftypes.String], "map":tftypes.Map[tftypes.String], "name":tftypes.String, "string":tftypes.String]]], "ref":tftypes.String, "repoURL":tftypes.String, "targetRevision":tftypes.String], "sources":tftypes.Tuple[tftypes.Object["chart":tftypes.String, "directory":tftypes.Object["exclude":tftypes.String, "include":tftypes.String, "jsonnet":tftypes.Object["extVars":tftypes.List[tftypes.Object["code":tftypes.Bool, "name":tftypes.String, "value":tftypes.String]], "libs":tftypes.List[tftypes.String], "tlas":tftypes.List[tftypes.Object["code":tftypes.Bool, "name":tftypes.String, "value":tftypes.String]]], "recurse":tftypes.Bool], "helm":tftypes.Object["fileParameters":tftypes.List[tftypes.Object["name":tftypes.String, "path":tftypes.String]], "ignoreMissingValueFiles":tftypes.Bool, "parameters":tftypes.List[tftypes.Object["forceString":tftypes.Bool, "name":tftypes.String, "value":tftypes.String]], "passCredentials":tftypes.Bool, "releaseName":tftypes.String, "skipCrds":tftypes.Bool, "valueFiles":tftypes.List[tftypes.String], "values":tftypes.String, "valuesObject":tftypes.DynamicPseudoType, "version":tftypes.String], "kustomize":tftypes.Object["commonAnnotations":tftypes.Map[tftypes.String], "commonAnnotationsEnvsubst":tftypes.Bool, "commonLabels":tftypes.Map[tftypes.String], "components":tftypes.List[tftypes.String], "forceCommonAnnotations":tftypes.Bool, "forceCommonLabels":tftypes.Bool, "images":tftypes.List[tftypes.String], "namePrefix":tftypes.String, "nameSuffix":tftypes.String, "namespace":tftypes.String, "patches":tftypes.List[tftypes.Object["options":tftypes.Map[tftypes.Bool], "patch":tftypes.String, "path":tftypes.String, "target":tftypes.Object["annotationSelector":tftypes.String, "group":tftypes.String, "kind":tftypes.String, "labelSelector":tftypes.String, "name":tftypes.String, "namespace":tftypes.String, "version":tftypes.String]]], "replicas":tftypes.List[tftypes.Object["count":tftypes.String, "name":tftypes.String]], "version":tftypes.String], "path":tftypes.String, "plugin":tftypes.Object["env":tftypes.List[tftypes.Object["name":tftypes.String, "value":tftypes.String]], "name":tftypes.String, "parameters":tftypes.List[tftypes.Object["array":tftypes.List[tftypes.String], "map":tftypes.Map[tftypes.String], "name":tftypes.String, "string":tftypes.String]]], "ref":tftypes.String, "repoURL":tftypes.String, "targetRevision":tftypes.String]]]

goroutine 65 [running]: github.com/hashicorp/terraform-plugin-go/tftypes.NewValue(...) github.com/hashicorp/[email protected]/tftypes/value.go:278 github.com/hashicorp/terraform-provider-kubernetes/manifest/morph.DeepUnknown({0x251fbf8, 0xc00446e090?}, {{0x251fbf8?, 0xc005232450?}, {0x1d79ca0?, 0xc000c8a3d8?}}, 0xc000dd54a0) github.com/hashicorp/terraform-provider-kubernetes/manifest/morph/scaffold.go:86 +0x19ae github.com/hashicorp/terraform-provider-kubernetes/manifest/morph.DeepUnknown({0x251f760, 0xc00446e0c0?}, {{0x251f760?, 0xc0053f1080?}, {0x1ea6460?, 0xc005182300?}}, 0xc000dd4240) github.com/hashicorp/terraform-provider-kubernetes/manifest/morph/scaffold.go:33 +0x1cb4 github.com/hashicorp/terraform-provider-kubernetes/manifest/morph.DeepUnknown({0x251f760, 0xc00446e0f0?}, {{0x251f760?, 0xc00550ee70?}, {0x1ea6460?, 0xc005049260?}}, 0xc0062af368) github.com/hashicorp/terraform-provider-kubernetes/manifest/morph/scaffold.go:33 +0x1cb4 github.com/hashicorp/terraform-provider-kubernetes/manifest/provider.(*RawProviderServer).ReadPluralDataSource(0xc000538780, {0x25187b8, 0xc000c852f0}, 0xc000c84f60) github.com/hashicorp/terraform-provider-kubernetes/manifest/provider/datasource.go:187 +0x1634 github.com/hashicorp/terraform-provider-kubernetes/manifest/provider.(*RawProviderServer).ReadDataSource(0x25187b8?, {0x25187b8, 0xc000c852f0}, 0xc000c84f60) github.com/hashicorp/terraform-provider-kubernetes/manifest/provider/datasource.go:28 +0x85 github.com/hashicorp/terraform-plugin-mux/tf5muxserver.(*muxServer).ReadDataSource(0x25187f0?, {0x25187b8?, 0xc000c84ff0?}, 0xc000c84f60) github.com/hashicorp/[email protected]/tf5muxserver/mux_server_ReadDataSource.go:36 +0x193 github.com/hashicorp/terraform-plugin-go/tfprotov5/tf5server.(*server).ReadDataSource(0xc0005343c0, {0x25187b8?, 0xc000c845a0?}, 0xc0001541e0) github.com/hashicorp/[email protected]/tfprotov5/tf5server/server.go:688 +0x290 github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/tfplugin5._Provider_ReadDataSource_Handler({0x2113760?, 0xc0005343c0}, {0x25187b8, 0xc000c845a0}, 0xc0001d6900, 0x0) github.com/hashicorp/[email protected]/tfprotov5/internal/tfplugin5/tfplugin5_grpc.pb.go:572 +0x169 google.golang.org/grpc.(*Server).processUnaryRPC(0xc0001d2e00, {0x25187b8, 0xc000c84510}, {0x2520bc8, 0xc00021d500}, 0xc000e1e000, 0xc000c34c60, 0x35c0a00, 0x0) google.golang.org/[email protected]/server.go:1369 +0xe23 google.golang.org/grpc.(*Server).handleStream(0xc0001d2e00, {0x2520bc8, 0xc00021d500}, 0xc000e1e000) google.golang.org/[email protected]/server.go:1780 +0x1016 google.golang.org/grpc.(*Server).serveStreams.func2.1() google.golang.org/[email protected]/server.go:1019 +0x8b created by google.golang.org/grpc.(*Server).serveStreams.func2 in goroutine 16 google.golang.org/[email protected]/server.go:1030 +0x135

Error: The terraform-provider-kubernetes_v2.32.0_x5 plugin crashed!

This is always indicative of a bug within the plugin. It would be immensely helpful if you could report the crash with the plugin's maintainers so that it can be fixed. The output above should help diagnose the issue.

Makanist avatar Aug 14 '24 15:08 Makanist

I too am getting this error using terraform-provider-kubernetes_v2.32.0_x5

Error output

 goroutine 47 [running]:
  github.com/hashicorp/terraform-plugin-go/tftypes.NewValue(...)
        github.com/hashicorp/[email protected]/tftypes/value.go:278
  github.com/hashicorp/terraform-provider-kubernetes/manifest/morph.DeepUnknown({0x251fbf8, 0xc0044023c0?}, {{0x251fbf8?, 0xc004855500?}, {0x1d79ca0?, 0xc001ce60c0?}}, 0xc001c5a3a8)
        github.com/hashicorp/terraform-provider-kubernetes/manifest/morph/scaffold.go:86 +0x19ae
  github.com/hashicorp/terraform-provider-kubernetes/manifest/morph.DeepUnknown({0x251f760, 0xc0044cf380?}, {{0x251f760?, 0xc004787230?}, {0x1ea6460?, 0xc0046885d0?}}, 0xc001ce6c30)
        github.com/hashicorp/terraform-provider-kubernetes/manifest/morph/scaffold.go:33 +0x1cb4
  github.com/hashicorp/terraform-provider-kubernetes/manifest/morph.DeepUnknown({0x251f760, 0xc004688150?}, {{0x251f760?, 0xc0047ad350?}, {0x1ea6460?, 0xc0046883c0?}}, 0xc001ce6828)
        github.com/hashicorp/terraform-provider-kubernetes/manifest/morph/scaffold.go:33 +0x1cb4
  github.com/hashicorp/terraform-provider-kubernetes/manifest/provider.(*RawProviderServer).PlanResourceChange(0xc0002ee280, {0x25187b8, 0xc004118d50}, 0xc0004aaaa0)
        github.com/hashicorp/terraform-provider-kubernetes/manifest/provider/plan.go:395 +0x3553
  github.com/hashicorp/terraform-plugin-mux/tf5muxserver.(*muxServer).PlanResourceChange(0xc00023ee00, {0x25187b8?, 0xc004118a50?}, 0xc0004aaaa0)
        github.com/hashicorp/[email protected]/tf5muxserver/mux_server_PlanResourceChange.go:73 +0x2ad
  github.com/hashicorp/terraform-plugin-go/tfprotov5/tf5server.(*server).PlanResourceChange(0xc0002941e0, {0x25187b8?, 0xc004118000?}, 0xc0045d2700)
        github.com/hashicorp/[email protected]/tfprotov5/tf5server/server.go:825 +0x3f0
  github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/tfplugin5._Provider_PlanResourceChange_Handler({0x2113760?, 0xc0002941e0}, {0x25187b8, 0xc004118000}, 0xc0045d2680, 0x0)
        github.com/hashicorp/[email protected]/tfprotov5/internal/tfplugin5/tfplugin5_grpc.pb.go:500 +0x169
  google.golang.org/grpc.(*Server).processUnaryRPC(0xc000200e00, {0x25187b8, 0xc0040f3ef0}, {0x2520bc8, 0xc0006ba000}, 0xc0028c8000, 0xc001081d10, 0x35c09a0, 0x0)
        google.golang.org/[email protected]/server.go:1369 +0xe23
  google.golang.org/grpc.(*Server).handleStream(0xc000200e00, {0x2520bc8, 0xc0006ba000}, 0xc0028c8000)
        google.golang.org/[email protected]/server.go:1780 +0x1016
  google.golang.org/grpc.(*Server).serveStreams.func2.1()
        google.golang.org/[email protected]/server.go:1019 +0x8b
  created by google.golang.org/grpc.(*Server).serveStreams.func2 in goroutine 41
        google.golang.org/[email protected]/server.go:1030 +0x135

  Error: The terraform-provider-kubernetes_v2.32.0_x5 plugin crashed!

  This is always indicative of a bug within the plugin. It would be immensely
  helpful if you could report the crash with the plugin's maintainers so that it
  can be fixed. The output above should help diagnose the issue.


  exit status 1

sadminriley avatar Dec 05 '24 16:12 sadminriley

@rkul @Makanist Were either of you able to get this working on 2.32.0 at all yet? Trying to find the work around myself

sadminriley avatar Dec 05 '24 16:12 sadminriley

I just ran mine with a for_each, if it helps anyone

locals {
clusters = {
    stg  = "https://1234.foo.us-east-1.eks.amazonaws.com"
    prod = "https://5678.foo.us-east-1.eks.amazonaws.com"
  }
}

# This has to use a for_each statement with the local.clusters; throws provider error otherwise
# https://github.com/hashicorp/terraform-provider-kubernetes/issues/2440
resource "kubernetes_manifest" "app_set" {
  for_each = local.clusters

  manifest = {
    "apiVersion" : "argoproj.io/v1alpha1",
    "kind" : "ApplicationSet",
    "metadata" : {
      "name" : local.component,
      "namespace" : "argocd"
    },
    "spec" : {
      goTemplate : true,
      goTemplateOptions : ["missingkey=error"],
      "generators" : [
        {
          "list" : {
            "elements" : [
              {
                "cluster" : each.key,
                "url" : each.value
              }
            ]
          }
        }
      ],
      "template" : {
        "metadata" : {
          "name" : "${each.key}-${local.component}"
        },
        "spec" : {
          "project" : "default",
          "syncPolicy" : {
            "automated" : {
              "prune" : true,
              "selfHeal" : true
            },
            "syncOptions" : [
              "CreateNamespace=false"
            ]
          },
          "source" : {
            "repoURL" : "[email protected]:git-repo/${local.component}.git",
            "targetRevision" : "HEAD",
            "path" : "ops/kube/envs/${each.key}"
          },
          "destination" : {
            "server" : each.value,
            "namespace" : local.component
          }
        }
      }
    }
  }
}

sadminriley avatar Dec 05 '24 21:12 sadminriley