terraform-aws-eks-blueprints icon indicating copy to clipboard operation
terraform-aws-eks-blueprints copied to clipboard

feat: Add the capability to set the name of the IAM role to be created by IRSA module for add-on

Open florentio opened this issue 2 years ago • 2 comments

What does this PR do?

This PR is raised to add the capability to set the name of the IAM role to be created by IRSA module. No breaking changes are introduced by this PR to EKS blueprints.

IRSA creation through direct usage of irsa module

If you are using the IRSA module directly, you can define the name IAM Role to be created by IRSA module within irsa_iam_role_name (see FAQ.md for more details). This variable is optional

module "irsa" {
  source = "github.com/aws-ia/terraform-aws-eks-blueprints//modules/irsa"
  kubernetes_namespace       = "potato"
  kubernetes_service_account = "grafana"
  irsa_iam_policies          = [aws_iam_policy. grafana.arn]
  eks_cluster_id             = module.eks_blueprints.eks_cluster_id
  eks_oidc_provider_arn      = module.eks_blueprints.eks_oidc_provider_arn
  irsa_iam_role_name         = "my-super-grafana-irsa"
}

IRSA creation through for addons

If you are adding addons to EKS which required an IRSA and you would like to specify the name of IAM role to be created for IRSA, define this value to irsa_iam_role_name attribute of the helm_config map variable (see example below)

Before

locals {
  keda_helm_config = {}
}
module "keda" {
  count             = var.enable_keda ? 1 : 0
  source            = "./keda"
  helm_config       = local.keda_helm_config
  irsa_policies     = var.keda_irsa_policies
  manage_via_gitops = var.argocd_manage_add_ons
  addon_context     = local.addon_context
}

After

locals {
  keda_helm_config = {
    irsa_iam_role_name = "my-keda-role-for-irsa"
  }
}
module "keda" {
  count             = var.enable_keda ? 1 : 0
  source            = "./keda"
  helm_config       = local.keda_helm_config
  irsa_policies     = var.keda_irsa_policies
  manage_via_gitops = var.argocd_manage_add_ons
  addon_context     = local.addon_context
}

Related Issues

https://github.com/aws-ia/terraform-aws-eks-blueprints/issues/816

Motivation

  • There are many case where you would like to define the name of your IAM role in order to easily find, manage them, to match a standard across your organization or to reuse the role for another purpose.
  • To simplify user experience to consume addon moduleS which require IRSA.

More

  • [x] Yes, I have tested the PR using my local account setup (Provide any test evidence report under Additional Notes)
  • [ ] Yes, I have added a new example under examples to support my PR
  • [ ] Yes, I have created another PR for add-ons under add-ons repo (if applicable)
  • [x] Yes, I have updated the docs for this feature
  • [x] Yes, I ran pre-commit run -a with this PR

Note: Not all the PRs required examples and docs except a new pattern or add-on added.

For Moderators

  • [ ] E2E Test successfully complete before merge?

Additional Notes

florentio avatar Aug 29 '22 10:08 florentio

How does this resolve #816 (or even relate to it)?

bryantbiggs avatar Aug 30 '22 14:08 bryantbiggs

How does this resolve #816 (or even relate to it)?

There is a variable irsa_iam_role_name added here : https://github.com/aws-ia/terraform-aws-eks-blueprints/blob/015834f05273cab73310cec68386aebf7d64627b/modules/irsa/variables.tf#L29 by this PR https://github.com/aws-ia/terraform-aws-eks-blueprints/pull/832 in order to be able to define the name of the IAM role to be created by irsa module instead of the generated name inside the module.

helm-addon nodule is using the irsa module to create irsa for any helm charts which required irsa. the irsa configuration has to be sent in irsa_config variable of that module

In the current code of helm-addon, the name of the irsa role is set here : https://github.com/aws-ia/terraform-aws-eks-blueprints/blob/015834f05273cab73310cec68386aebf7d64627b/modules/kubernetes-addons/helm-addon/main.tf#L69 but and we can use the irsa_iam_role_name variable defined in helm-addon module to specify the name of the IAM role.

None of the helm modules are passing that variable to helm-addon module in their configuration.

The idea here is to use the irsa_config in order to pass the irsa_iam_role_name value if you want to set yourself the name of the irsa role as mentioned here https://github.com/aws-ia/terraform-aws-eks-blueprints/issues/816#issuecomment-1224251987

Short story, let's say that you are adding karpenter into your cluster with the karpenter addon here is an example of current karpenter usage

 # Optional  karpenter_helm_config local variable
  karpenter_helm_config = {
    name                       = "karpenter"
    chart                      = "karpenter"
     repository                 = "https://charts.karpenter.sh"
     version                    = "0.16.0"
     namespace                  = "karpenter"
     values = [templatefile("${path.module}/values.yaml", {
          eks_cluster_id       = var.eks_cluster_id,
          eks_cluster_endpoint = var.eks_cluster_endpoint,
         service_account_name = var.service_account_name,
         operating_system     = "linux"
    })]
}

module "eks_blueprints_kubernetes_addons" {
   source = "../../modules/kubernetes-addons"

   eks_cluster_id       = module.eks_blueprints.eks_cluster_id
   eks_cluster_endpoint = module.eks_blueprints.eks_cluster_endpoint
   eks_oidc_provider    = module.eks_blueprints.oidc_provider
   eks_cluster_version  = module.eks_blueprints.eks_cluster_version

   enable_karpenter                    = true
   karpenter_helm_config    = local.karpenter_helm_config
}

with that config, irsa role will be created by the second expression here : https://github.com/aws-ia/terraform-aws-eks-blueprints/blob/015834f05273cab73310cec68386aebf7d64627b/modules/irsa/main.tf#L27 since var.irsa_iam_role_name is empty. The helm-addon send an empty value here : https://github.com/aws-ia/terraform-aws-eks-blueprints/blob/015834f05273cab73310cec68386aebf7d64627b/modules/kubernetes-addons/helm-addon/main.tf#L69 to irsa module because the karpenter module here https://github.com/aws-ia/terraform-aws-eks-blueprints/blob/015834f05273cab73310cec68386aebf7d64627b/modules/kubernetes-addons/karpenter/main.tf#L1 which didnt defined the value of irsa_iam_role_name.

Instead of adding the irsa_iam_role_name variable into all addons and passed it to helm-addon like this in order to set the name of the irsa role

module "helm_addon" {
  source             = "../helm-addon"
  manage_via_gitops  = var.manage_via_gitops
  helm_config        = local.helm_config
  set_values         = local.set_values
  irsa_config        = local.irsa_config
  addon_context      = var.addon_context
  irsa_iam_role_name = var.irsa_iam_role_name
}

the proposal here is to add optional attribute :

  • irsa_iam_role_name to helm_config map variable defined for addon's modules
  • irsa_iam_role_name to irsa_config map variable defined in helm-addon module

Let's say you want to have the name of the irsa role for karpenter to be equals to my-karpenter-role

here is the an example of current karpenter usage

  # Optional  karpenter_helm_config local variable
  karpenter_helm_config = {
     irsa_iam_role_name = 'my-karpenter-role'
     name                       = "karpenter"
     chart                      = "karpenter"
     repository                 = "https://charts.karpenter.sh"
     version                    = "0.16.0"
     namespace                  = "karpenter"
     values = [templatefile("${path.module}/values.yaml", {
          eks_cluster_id       = var.eks_cluster_id,
          eks_cluster_endpoint = var.eks_cluster_endpoint,
         service_account_name = var.service_account_name,
         operating_system     = "linux"
    })]
  }
  
  module "eks_blueprints_kubernetes_addons" {
  	source = "../../modules/kubernetes-addons"

  	eks_cluster_id       = module.eks_blueprints.eks_cluster_id
  	eks_cluster_endpoint = module.eks_blueprints.eks_cluster_endpoint
  	eks_oidc_provider    = module.eks_blueprints.oidc_provider
  	ks_cluster_version  = module.eks_blueprints.eks_cluster_version

  	enable_karpenter                    = true
  	karpenter_helm_config    = local.karpenter_helm_config
}

This configuration will be call karpenter module here https://github.com/aws-ia/terraform-aws-eks-blueprints/blob/015834f05273cab73310cec68386aebf7d64627b/modules/kubernetes-addons/main.tf#L235 and the module call with be like this:

module "karpenter" {
  count                     = 1 
  source                    = "./karpenter"
  helm_config               = {
     irsa_iam_role_name = 'my-karpenter-role'
     name                       = "karpenter"
     chart                      = "karpenter"
     repository                 = "https://charts.karpenter.sh"
     version                    = "0.16.0"
     namespace                  = "karpenter"
     values = [templatefile("${path.module}/values.yaml", {
          eks_cluster_id       = var.eks_cluster_id,
          eks_cluster_endpoint = var.eks_cluster_endpoint,
         service_account_name = var.service_account_name,
         operating_system     = "linux"
    })]
  }
  irsa_policies             = []
  node_iam_instance_profile = ""
  manage_via_gitops         = false
  addon_context             = local.addon_context
}

Karpenter module will then call the helm-addon module by passing irsa_config defined below

irsa_config = {
    kubernetes_namespace              = local.helm_config["namespace"]
    kubernetes_service_account        = local.service_account_name
    create_kubernetes_namespace       = try(local.helm_config["create_namespace"], true)
    create_kubernetes_service_account = true
    irsa_iam_role_name                = try(var.helm_config.irsa_iam_role_name, "")
    irsa_iam_policies                 = concat([aws_iam_policy.karpenter.arn], var.irsa_policies)
}

As we can see since this irsa_iam_role_name = try(var.helm_config.irsa_iam_role_name, "") will be equals to irsa_iam_role_name = 'my-karpenter-role' Finally the helm-addon module while calling irsa module will pass the iam role name here https://github.com/aws-ia/terraform-aws-eks-blueprints/blob/f7c93c8593bcd2ff287fbc77feaab0062a2498cb/modules/kubernetes-addons/helm-addon/main.tf#L69

If you don't want to specifcy the role of iam to be created by irsa, you just have to not set the irsa_iam_role_name since it's optional.

This is applicable to the addon which has irsa.

Also in the PR, i do remove irsa config in variable for helm chart doesnot required it to make it cleaner.

florentio avatar Aug 30 '22 15:08 florentio

Thank you for the PR - we have plans to change the way that IRSA and helm-addons are provisioned to improve upon a number of factors and we will be addressing all IRSA/helm addon changes in that module

bryantbiggs avatar Oct 06 '22 12:10 bryantbiggs

Ok sounds good. Shoud i close it ?

florentio avatar Oct 06 '22 16:10 florentio

@bryantbiggs : Do you have an ETA for the rework you refer to? This specific PR does address a real problem, so if the rework is months in the future, maybe the PR can be used in the meanwhile.

vprus avatar Oct 13 '22 13:10 vprus

We are maintaining a fork of this repo with our own implementation of this. I would love this PR brought in for the interim (days to weeks to months) while a larger effort is in place.

sbrinkerhoff avatar Oct 21 '22 14:10 sbrinkerhoff