terraform-provider-helm icon indicating copy to clipboard operation
terraform-provider-helm copied to clipboard

Error: Blocks of type "kubernetes" are not expected here.

Open lwu-st opened this issue 5 months ago • 29 comments

⚠ Note From Maintainers: v3.0.0 breaking changes

If you have landed on this issue after upgrading the Helm provider to v3.0.0, please see our Upgrade Guide as this is a major release which features breaking changes to the Terraform protocol and to the syntax for attributes which were previously SDKv2 blocks. If you do not want to migrate your configuration to the new syntax please pin your provider to v2.17.0.

If you prefer, you can also ask your question in the Kubernetes community Slack channel #terraform-providers. (Sign up here)

Terraform version, Kubernetes provider version and Kubernetes version

Terraform version: 1.11.2
Helm Provider version: 3.0.0
Kubernetes version: 2.37.1, and/or 2.25.1

Terraform configuration

terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = ">= 4.33.0"
    }
    kubernetes = {
      source  = "hashicorp/kubernetes"
      version = ">= 2.37.1"
    }
    helm = {
      source  = "hashicorp/helm"
      version = ">= 3.0.0"
    }
  }
}

provider "kubernetes" {
  host                   = ...    = ...
  client_key             = ...
  cluster_ca_certificate = ...
}

provider "helm" {
  kubernetes {
    host                   = ...    
    client_key             = ...
    cluster_ca_certificate = ...
  }
}

Question

when running tf refresh, I seem to be getting this error. It's odd because even a few minutes ago, it was succeeding. I've tried updating my kubernetes and helm providers to the latest versions as well, and it results in the same error.

Failed to run tf refresh. Output: Acquiring state lock. This may take a few moments...
Releasing state lock. This may take a few moments...
Γò╖
Γöé Error: Unsupported block type
Γöé
Γöé   on modules\stamp\providers.tf line 26, in provider "helm":
Γöé   26:   kubernetes {
Γöé
Γöé Blocks of type "kubernetes" are not expected here. Did you mean to define
Γöé argument "kubernetes"? If so, use the equals sign to assign it a value.
Γò╡

Any advice on troubleshooting this would be greatly appreciated!

lwu-st avatar Jun 18 '25 17:06 lwu-st

Getting the same issue on my clusters now:

Terraform v1.8.5
on linux_amd64
Initializing plugins and modules...
╷
│ Error: Unsupported block type
│
│   on providers.tf line 24, in provider "helm":
│   24:   kubernetes {
│
│ Blocks of type "kubernetes" are not expected here. Did you mean to define
│ argument "kubernetes"? If so, use the equals sign to assign it a value.
╵
Operation failed: failed running terraform plan (exit 1)

ffh-andrew avatar Jun 18 '25 18:06 ffh-andrew

@lwu-st your helm provider is at version 3. There are some breaking changes explained here

For this case you basically need to change your helm provider blocks from:

provider "helm" {
  kubernetes {
    host                   = ...    = ...
    client_key             = ...
    cluster_ca_certificate = ...
  }
}

to

provider "helm" {
  kubernetes  = {
    host                   = ...    = ...
    client_key             = ...
    cluster_ca_certificate = ...
  }
}

Note the = at the second line in the second code block

ssro avatar Jun 18 '25 18:06 ssro

We are also encountering this issue during terraform validate. Just started today, code worked yesterday. Issue is also identified by VSCode's Terraform linting engine.

ricksheets-chipotle avatar Jun 18 '25 18:06 ricksheets-chipotle

@lwu-st your helm provider is at version 3. There are some breaking changes explained here

For this case you basically need to change your helm provider blocks from:

provider "helm" {
  kubernetes {
    host                   = ...    = ...
    client_key             = ...
    cluster_ca_certificate = ...
  }
}

to

provider "helm" {
  kubernetes  = {
    host                   = ...    = ...
    client_key             = ...
    cluster_ca_certificate = ...
  }
}

Note the = at the second line in the second code block

Even with this fix, it seems other things in this module is just broked on version 3.0.0, for example:

Error: Unsupported block type │ │ on .terraform/modules/eks.helm_prometheus/main.tf line 49, in resource "helm_release" "main": │ 49: dynamic "set_sensitive" { │ │ Blocks of type "set_sensitive" are not expected here. ╵ ╷ │ Error: Unsupported block type │ │ on .terraform/modules/eks.helm_prometheus/main.tf line 59, in resource "helm_release" "main": │ 59: dynamic "postrender" { │ │ Blocks of type "postrender" are not expected here.

Using version 2.17.0 everything works fine.

memorais avatar Jun 18 '25 18:06 memorais

Hi @ssro and @lwu-st,

Thanks for raising this! Starting with Helm provider v3.0.0, the provider has been migrated to the Terraform Plugin Framework, which introduces breaking changes—including the removal of inline Kubernetes configuration inside the helm provider.

here is the correct usage in helm v3.0.0

provider "helm" {
  kubernetes = {
    config_path = "~/.kube/config"
  }
}

jaylonmcshan19-x avatar Jun 18 '25 18:06 jaylonmcshan19-x

@lwu-st your helm provider is at version 3. There are some breaking changes explained here For this case you basically need to change your helm provider blocks from:

provider "helm" {
  kubernetes {
    host                   = ...    = ...
    client_key             = ...
    cluster_ca_certificate = ...
  }
}

to

provider "helm" {
  kubernetes  = {
    host                   = ...    = ...
    client_key             = ...
    cluster_ca_certificate = ...
  }
}

Note the = at the second line in the second code block

Even with this fix, it seems other things in this module is just broked on version 3.0.0, for example:

Error: Unsupported block type │ │ on .terraform/modules/eks.helm_prometheus/main.tf line 49, in resource "helm_release" "main": │ 49: dynamic "set_sensitive" { │ │ Blocks of type "set_sensitive" are not expected here. ╵ ╷ │ Error: Unsupported block type │ │ on .terraform/modules/eks.helm_prometheus/main.tf line 59, in resource "helm_release" "main": │ 59: dynamic "postrender" { │ │ Blocks of type "postrender" are not expected here.

Using version 2.17.0 everything works fine.

The set_sensitive are in those breaking changes too (see the link i've sent earlier). The correct way now is:

set_sensitive = [
    {
      name  = "name"
      value = "value"
    }
  ]

ssro avatar Jun 18 '25 18:06 ssro

@memorais

For set_sensitive this is the config you should use

set_sensitive = [
  {
    name  = ""
    value = ""
  }
]

For Postrender this is the config you should use

postrender =  {
   binary_path = ""
   args        = []
}

jaylonmcshan19-x avatar Jun 18 '25 18:06 jaylonmcshan19-x

@memorais

For set_sensitive this is the config you should use

set_sensitive = [
  {
    name  = "%q"
    value = "%q"
  }
]

For Postrender this is the config you should use

postrender =  {
			        binary_path = %q
				args        = [%s]
			}

@jaylonmcshan19-x we'd like to have a better documentation about v3. For example I'm still trying to figure out how to use values block, since now I get errors

With a value block like this

  values = [
    file("./path/to/local/nginx.yaml")
  ]

i get this error

  │ Error: Failed to unmarshal prior state
  │
  │   with helm_release.nginx-ingress,
  │   on charts.tf line 383, in resource "helm_release" "nginx-ingress":
  │  383: resource "helm_release" "nginx-ingress" {
  │
  │ AttributeName("values"): unsupported type json.Delim sent as tftypes.String
  ╵

  exit status 1

Any guidance is appreciated 🙏

ssro avatar Jun 18 '25 18:06 ssro

I have a similar issue as above, except my values block looks something like:

values = [
  <<EOF
  foo: bar
  EOF
]

I checked the migration guide, and found no mention of changes to values.

dvdmuckle avatar Jun 18 '25 18:06 dvdmuckle

Hi @ssro can you try this config >

values = [
  yamlencode(yamldecode(file("./path/to/nginx.yaml")))
]

jaylonmcshan19-x avatar Jun 18 '25 18:06 jaylonmcshan19-x

yamlencode(yamldecode(file("./path/to/nginx.yaml")))

@jaylonmcshan19-x I get the same error unfortunately

ssro avatar Jun 18 '25 18:06 ssro

We are also experiencing issues umarshelling prior state. Trace log attached, but does not offer much more detail:

2025-06-18T20:42:45.284+0200 [TRACE] provider.terraform-provider-helm_v3.0.0_x5: Called provider defined StateUpgrader: @caller=github.com/hashicorp/[email protected]/internal/fwserver/server_upgraderesourcestate.go:206 @module=sdk.framework tf_provider_addr=registry.terraform.io/hashicorp/helm tf_req_id=b196484d-9ef8-6c66-8d0d-6729fafd92ab tf_resource_type=helm_release tf_rpc=UpgradeResourceState timestamp="2025-06-18T20:42:45.284+0200"
2025-06-18T20:42:45.284+0200 [TRACE] provider.terraform-provider-helm_v3.0.0_x5: Received downstream response: tf_resource_type=helm_release tf_rpc=UpgradeResourceState tf_req_duration_ms=0 @module=sdk.proto diagnostic_error_count=1 diagnostic_warning_count=0 tf_proto_version=6.9 @caller=github.com/hashicorp/[email protected]/tfprotov6/internal/tf6serverlogging/downstream_request.go:42 tf_req_id=b196484d-9ef8-6c66-8d0d-6729fafd92ab tf_provider_addr=registry.terraform.io/hashicorp/helm timestamp="2025-06-18T20:42:45.284+0200"
2025-06-18T20:42:45.284+0200 [ERROR] provider.terraform-provider-helm_v3.0.0_x5: Response contains error diagnostic: @module=sdk.proto diagnostic_severity=ERROR diagnostic_summary="Failed to unmarshal prior state" tf_proto_version=6.9 tf_req_id=b196484d-9ef8-6c66-8d0d-6729fafd92ab @caller=github.com/hashicorp/[email protected]/tfprotov6/internal/diag/diagnostics.go:58 tf_rpc=UpgradeResourceState tf_resource_type=helm_release diagnostic_detail="AttributeName(\"values\"): unsupported type json.Delim sent as tftypes.String" tf_provider_addr=registry.terraform.io/hashicorp/helm timestamp="2025-06-18T20:42:45.284+0200"
2025-06-18T20:42:45.284+0200 [TRACE] provider.terraform-provider-helm_v3.0.0_x5: Served request: @caller=github.com/hashicorp/[email protected]/tfprotov6/tf6server/server.go:833 @module=sdk.proto tf_proto_version=6.9 tf_req_id=b196484d-9ef8-6c66-8d0d-6729fafd92ab tf_resource_type=helm_release tf_rpc=UpgradeResourceState tf_provider_addr=registry.terraform.io/hashicorp/helm timestamp="2025-06-18T20:42:45.284+0200"
2025-06-18T20:42:45.284+0200 [ERROR] vertex "module.rabbitmq.helm_release.rabbitmq" error: Failed to unmarshal prior state
2025-06-18T20:42:45.284+0200 [TRACE] vertex "module.rabbitmq.helm_release.rabbitmq": visit complete, with errors
2025-06-18T20:42:45.284+0200 [TRACE] dag/walk: upstream of "root" errored, so skipping
2025-06-18T20:42:45.284+0200 [TRACE] vertex "module.rabbitmq.helm_release.rabbitmq (expand)": dynamic subgraph encountered errors: Failed to unmarshal prior state
2025-06-18T20:42:45.284+0200 [ERROR] vertex "module.rabbitmq.helm_release.rabbitmq (expand)" error: Failed to unmarshal prior state
2025-06-18T20:42:45.284+0200 [TRACE] vertex "module.rabbitmq.helm_release.rabbitmq (expand)": visit complete, with errors
2025-06-18T20:42:45.284+0200 [TRACE] dag/walk: upstream of "provider[\"registry.terraform.io/hashicorp/helm\"] (close)" errored, so skipping
2025-06-18T20:42:45.284+0200 [TRACE] dag/walk: upstream of "module.rabbitmq (close)" errored, so skipping

sprinthive-kieran avatar Jun 18 '25 18:06 sprinthive-kieran

I found this issue by searching GitHub and I have the same problem in v3 as the last few messages. All of my helm_releases give the following error after upgrading to v3 and following the migration guide:

╷
│ Warning: Failed to decode resource from state
│ 
│ Error decoding "helm_release.pgadmin" from prior state: missing expected {
╵
╷
│ Error: Failed to unmarshal prior state
│ 
│   with helm_release.pgadmin,
│   on pgadmin.tf line 10, in resource "helm_release" "pgadmin":
│   10: resource "helm_release" "pgadmin" {
│ 
│ AttributeName("values"): unsupported type json.Delim sent as tftypes.String

In my resources, I always provide a file inside a list, like:

  values = [
    file("pgadmin.yaml")
  ]

Seems to me like the state upgrader tries todo something funky in these cases...

RemcodM avatar Jun 18 '25 19:06 RemcodM

Facing the same issue too with the latest v3 release.

│ Error: Failed to unmarshal prior state
│
│   with helm_release.main,
│   on chart.tf line 30, in resource "helm_release" "main":
│   30: resource "helm_release" "main" {
│
│ AttributeName("values"): unsupported type json.Delim sent as tftypes.String

smonfort-cacd2 avatar Jun 18 '25 19:06 smonfort-cacd2

I managed to fix the above issue by following the steps below:

  • revert to version 2.17.0
  • terraform state rm helm_release.cilium
  • upgrade to 3.0.0
  • terraform import helm_release.cilium kube-system/cilium
  • fix the repository attribute
    terraform state pull > state.json
    # edit the repository attribute from null to "https://helm.cilium.io"
    terraform state push -force state.json
    

nasenov avatar Jun 18 '25 19:06 nasenov

It looks like the state upgrader code has the wrong type for values in the schema, here:

https://github.com/hashicorp/terraform-provider-helm/blob/71b0a33707c7fc87926290a7181681172bffe284/helm/resource_helm_release_stateupgrader.go#L106

and here:

https://github.com/hashicorp/terraform-provider-helm/blob/71b0a33707c7fc87926290a7181681172bffe284/helm/resource_helm_release_stateupgrader.go#L252

It should be tftypes.List{ElementType: tftypes.String} not tftypes.String.

@jaylonmcshan19-x I've opened a PR with a hotfix and a new test to cover the error in #1638.

The workaround @nasenov suggested is an option to unstick blocked workflows in the meantime.

jrhouston avatar Jun 18 '25 19:06 jrhouston

@smonfort-cacd2 @RemcodM We've just cut a hotfix for the AttributeName("values"): unsupported type json.Delim sent as tftypes.String issue. It should be available on the Terraform registry shortly! Thank you for reporting the issue :)

jaylonmcshan19-x avatar Jun 18 '25 20:06 jaylonmcshan19-x

Can confirm the hotfix has fixed the issue with values. Thank you!

dvdmuckle avatar Jun 19 '25 01:06 dvdmuckle

for set_sensitive values the actual values are not stored in the k8s secrets for example:

set_sensitive = [
    {
      name  = "config.secret.argocdServerAdminPassword"
      value = random_password.argo_admin.result
    }
  ]

planned value:


set_sensitive              = [
{
  name  = "config.secret.argocdServerAdminPassword"
  value = (sensitive value)
},
]

Post apply when I try to decode the secret stored:

kubectl -n argocd get secret argocd-secret -o jsonpath="{.data.clearPassword}" | base64 -d

(sensitive value)% 

the actual values are not replaced, it stores (sensitive value) in the secrets

raghavanrrs avatar Jun 19 '25 04:06 raghavanrrs

Thank you! The hotfix resolved the values issue

ssro avatar Jun 19 '25 04:06 ssro

It feels like this v3 breaking change wasn't tested at all. The "(sensitive value)" thing is so insanely broken

abatilo avatar Jun 19 '25 05:06 abatilo

Can confirm the hotfix has fixed the issue with values. Thank you!

I can confirm that the values issue is fixed with 3.0.1. Sadly, I now encounter a new bug reported in #1641. It feels to me that v3 is just not there yet and I should wait for a few more bugfixes...

RemcodM avatar Jun 19 '25 06:06 RemcodM

for set_sensitive values the actual values are not stored in the k8s secrets for example:

set_sensitive = [
    {
      name  = "config.secret.argocdServerAdminPassword"
      value = random_password.argo_admin.result
    }
  ]

planned value:


set_sensitive              = [
{
  name  = "config.secret.argocdServerAdminPassword"
  value = (sensitive value)
},
]

Post apply when I try to decode the secret stored:

kubectl -n argocd get secret argocd-secret -o jsonpath="{.data.clearPassword}" | base64 -d

(sensitive value)% 

the actual values are not replaced, it stores (sensitive value) in the secrets

I can confirm, that the same issue with (sensitive value) in the k8s secrets occurs to us.

cnocula-peg avatar Jun 19 '25 09:06 cnocula-peg

https://github.com/hashicorp/terraform-provider-helm/pull/1644/files

This PR fixes the (set sensitive) problem

abatilo avatar Jun 19 '25 14:06 abatilo

I am probably to late to ask this question, but was it really worth breaking the support for configuration blocks? What is the benefit of doing so?

I understand why it is interesting to migrate to the plugin framework, but it has been a while that blocks have been un-deprecated: https://github.com/hashicorp/terraform-plugin-framework/issues/329 So I think it would have been better to keep blocks and keep the compatibility for practitioners.

lemaitre-aneo avatar Jun 19 '25 20:06 lemaitre-aneo

For anyone that needs an immediate fix (❗) to the secrets issue before that PR is merged/built/released,

set = [
  {
    name = "config.secret"
    value = sensitive(some.resource.secret)
  }
]

translates to the same behavior, as far as I can tell.

sikha-root avatar Jun 20 '25 10:06 sikha-root

We were also burnt by this:

╷
│ Error: Unsupported block type
│ 
│   on provider.tf line 23, in provider "helm":
│   23:   kubernetes {
│ 
│ Blocks of type "kubernetes" are not expected here. Did you mean to define argument "kubernetes"? If so, use the equals sign to assign it a value.
╵

Exited with code exit status 1

😢

jd-sandk avatar Jun 20 '25 15:06 jd-sandk

It looks like the documentation needs to be updated as well. https://registry.terraform.io/providers/hashicorp/helm/latest/docs#credentials-config mentions using a block while this https://registry.terraform.io/providers/hashicorp/helm/latest/docs#exec-plugins mentions using an object with =.

AnshumanTripathi avatar Jun 20 '25 16:06 AnshumanTripathi

@AnshumanTripathi Thanks for catching the documentation mishap! Will make a PR shortly with the correct configuration!

jaylonmcshan19-x avatar Jun 20 '25 16:06 jaylonmcshan19-x

Hello! Good to know this changed - should've read the docs to begin with.

However: when I fix the kubernetes block, turning it into a nested object, VS Code (VS Codium) HashiCorp Terraform Plugin (release from 2025-06-20, v2.34.5) seems to not like this change even though the CLI does.

Is there something else I should upgrade?

divStar avatar Jun 22 '25 20:06 divStar