Facing context deadline exceeded while updating helm_release

[farhan-sayeed]

Terraform, Provider, Kubernetes and Helm Versions

Terraform version: v1.5.3
AWS Provider: >= 4.2.0
Helm Provider: v2.12.1
Kubernetes version: v2.25.2

Affected Resource(s)

• helm_release

Terraform Configuration Files

resource "helm_release" "ebs_csi_driver" {
  name       = "aws-ebs-csi-driver"
  namespace  = "kube-system"
  repository = "https://kubernetes-sigs.github.io/aws-ebs-csi-driver"
  chart      = "aws-ebs-csi-driver"
  version    = var.ebs_csi_driver_helm_version

  recreate_pods     = var.helm_recreate_pods
  atomic            = var.helm_atomic_creation
  cleanup_on_fail   = var.helm_cleanup_on_fail
  wait              = var.helm_wait_for_completion
  wait_for_jobs     = var.helm_wait_for_jobs
  timeout           = var.helm_timeout_seconds
  max_history       = var.helm_max_history
  verify            = var.helm_verify
  keyring           = var.helm_keyring
  reuse_values      = var.helm_reuse_values
  reset_values      = var.helm_reset_values
  force_update      = var.helm_force_update
  replace           = var.helm_replace
  create_namespace  = var.helm_create_namespace
  dependency_update = var.helm_dependency_update
  skip_crds         = var.helm_skip_crds

  set {
    name  = "controller.serviceAccount.create"
    value = "false"
  }

  set {
    name  = "controller.serviceAccount.name"
    value = kubernetes_service_account.ebs_csi_driver.metadata[0].name
  }

  set {
    name  = "image.repository"
    value = var.eks_core_ebs_plugin_image_repo
  }

  set {
    name  = "image.tag"
    value = var.eks_core_ebs_plugin_image_tag
  }

  set {
    name  = "sidecars.provisioner.image.repository"
    value = var.eks_core_ebs_csi_provisioner_image_repo
  }

  set {
    name  = "sidecars.provisioner.image.tag"
    value = var.eks_core_ebs_csi_provisioner_image_tag
  }

  set {
    name  = "sidecars.attacher.image.repository"
    value = var.eks_core_ebs_csi_attacher_image_repo
  }

  set {
    name  = "sidecars.attacher.image.tag"
    value = var.eks_core_ebs_csi_attacher_image_tag
  }

  set {
    name  = "sidecars.livenessProbe.image.repository"
    value = var.eks_core_ebs_livenessprobe_image_repo
  }

  set {
    name  = "sidecars.livenessProbe.image.tag"
    value = var.eks_core_ebs_livenessprobe_image_tag
  }

  set {
    name  = "sidecars.resizer.image.repository"
    value = var.eks_core_ebs_csi_resizer_image_repo
  }

  set {
    name  = "sidecars.resizer.image.tag"
    value = var.eks_core_ebs_csi_resizer_image_tag
  }

  set {
    name  = "sidecars.nodeDriverRegistrar.image.repository"
    value = var.eks_core_ebs_node_driver_registrar_image_repo
  }

  set {
    name  = "sidecars.nodeDriverRegistrar.image.tag"
    value = var.eks_core_ebs_node_driver_registrar_image_tag
  }

  set {
    name  = "useOldCSIDriver"
    value = "true"
  }
}

Debug Output

│ Error: an error occurred while rolling back the release. original upgrade error: context deadline exceeded: release aws-ebs-csi-driver failed: Unauthorized │ │ with helm_release.ebs_csi_driver,

Steps to Reproduce

1. terraform apply
2. Checked with helm history command & it too shows context deadline exceeded & then after it tries rolling back to the last deployed revision

Expected Behavior

To upgrade the helm _release successfully without any errors.

Actual Behavior

While upgrading EBS CSI driver add-on version using terraform, it's throws context deadline exceeded error intermittently. Sometimes it works fine upgrading the version but recently I've been facing the same deadline exceeded error frequently.

Workarounds I tried

• tried increasing the timeout argument from default value 300 sec to 600 sec but this didn't helped me permanently. As I said, it's throwing such error intermittently & it becomes difficult to identify the cause
• Keep waiting for the helm rollback to get deployed successfully & then tried running terraform apply but still it throws the same error. Didn't understand if this's some kind of bug or a known issue

Important Factoids

Sometimes I can see it's throwing somewhat different error while deploying through terraform

│ Error: an error occurred while rolling back the release. original upgrade error: context deadline exceeded: release aws-ebs-csi-driver failed: Unauthorized

I don't understand why it started throwing such issues recently from last 3-4 days whilst we haven't made any configuration or provider changes

References

• https://github.com/helm/helm/issues/8675

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

[alexsomesan]

@farhan-sayeed is the original version that you are trying to upgrade from also installed through Terraform?

[iBrandyJackson]

Without further feedback to proceed, we're closing this ticket to maintain clarity in our issue tracker. This closure is purely procedural; we're still interested in addressing your concerns.

Should this issue persist or if you have additional information to share, please feel free to open a new ticket. Your contributions are valuable to us, and we're keen to assist wherever possible.