[QUESTION] Bring your own IAM roles for EKS and K8s add-ons

[mikeinton]

Please describe your question here

Hi there, our team operates in an AWS account that has a few restrictions on IAM resources.

• IAM role & policy names must be prefixed with a specific string
• If the IAM role can not be prefixed, it must be created in a specific role path
  • Example: "arn:aws:iam::xxx:role/{SOMEPATH}/MyRoleName"
• No tagging is allowed on any IAM resources

Due to these restrictions, we need to maintain IAM roles/permissions manually vs having most tools create and manage them for us.

The main EKS Blueprint module and managed node groups supports bringing your own IAM role in the form of.

create_iam_role = false
iam_role_arn = {myPrecreatedRoleARN}

But for any other EKS or K8s add-ons that require IAM roles/permissions this doesn't seem possible.

More

• [ x] Yes, I have checked the repo for existing issues before raising this question

[bryantbiggs]

Apologies - thought this was for cluster resources but I see this is for addons

[florentio]

Hello @mikeinton, Regarding addons, IAM Role usage is most for IRSA purpose (addons which need irsa config). So, I guess this issue is about providing the IRSA role to the addons instead of creating it inside the module call by the built-in irsa module.

Please correct me if I'm wrong.

If so, the only one gotcha will be to handle yourself the assume_role_policy of the role as required for IRSA which needs oidc provider, namespace etc.. and those information are available in the eks cluster configuration.

[mikeinton]

Hi @florentio that is correct, if the IRSA module is responsible for creating IAM resources for addons then could an alternative solution be to have the IRSA module create those IAM resources that follow a specific naming convention?

Here's an example, for the AWS LB Controller it seems that the following role will be created.

{clustername}-aws-load-balancer-controller-sa-irsa

This breaks for us because due to corporate AWS governance restrictions, we can't create any IAM resources unless they follow these guidelines.

• Must be prefixed with a specific string like so the resulting role would look something like CustomerManaged_{clustername}-aws-load-balancer-controller-sa-irsa

Or

• Must be created in a designated IAM role path like arn:aws:iam::xxx:role/{SOMEPATH}/{clustername}-aws-load-balancer-controller-sa-irsa"

Also note that our limitations extend to the IAM permissions policy as well where I don't see policy path as being an option so we would also need the resulting permissions policy to be prefixed and to look something like arn:aws:iam::xxx:policy/CustomerManaged_{clustername}-lb-irsa

Lastly, I didn't capture this in my initial post but we would also have to specify an IAM Permission Boundary for any roles that would be created during this workflow.

Appreciate the help and feedback, please let me know if you have any additional questions.

[florentio]

Hello @mikeinton

Thank for the reply, looks like the solution is covered by this PR https://github.com/aws-ia/terraform-aws-eks-blueprints/pull/908 related to this issue here https://github.com/aws-ia/terraform-aws-eks-blueprints/issues/816

This PR allows to define the name of the role related to IRSA. So you will can set the name you want according to your standard or naming convention.

Regarding the role path, you can use irsa_iam_role_path and for permission boundaries, you can set is within the irsa_iam_permissions_boundary variable

Both two variables will be apply to all IRSA roles created for all the addons enabled in your config. So if you want a separate path, permissions boundary for each addons enabled in your config, we will need a PR to provide that feature (but not sure if this feature is relevant).

Regarding the default policy provided by each IRSA, the policy name attached to the IRSA of each addons is builtin, also unfortunately there is no path attribute also to override. Those 2 statements can be fixed within a PR as well.

To summarize, once this PR is merged you can :

• set the name of each IRSA role to any value you want
• define the path of all IRSA roles once
• define the permissions boundary for all iIRSA role once

Let's go through an example. You want to add two addons to your cluster

• AWS LB Controller with the role name to equals to CustomerManaged_{clustername}-awesome-aws-lb-controller-irsa
• Cluster autoscaler with the role name to equals to my-cluster-autoscaler-role-for-irsa and both roles should be create with the path equals to my-awesome-path and you want to add a permission boundary

below is a terraform addons config

module "eks_blueprints_kubernetes_addons" {
  source = "../../../modules/kubernetes-addons"

  eks_cluster_id       = module.eks_blueprints.eks_cluster_id
  eks_cluster_endpoint = module.eks_blueprints.eks_cluster_endpoint
  eks_oidc_provider    = module.eks_blueprints.oidc_provider
  eks_cluster_version  = module.eks_blueprints.eks_cluster_version

  irsa_iam_role_path            = "my-awesome-path"            # all IRSA roles with have this as path
  irsa_iam_permissions_boundary = "<your-permission-boundary>" # all IRSA roles with have this as permissions boundary

  # Add-ons
  enable_aws_load_balancer_controller = true
  enable_cluster_autoscaler           = true

  aws_load_balancer_controller_helm_config = {
    irsa_iam_role_name = "CustomerManaged_${module.eks_blueprints.eks_cluster_id}-awesome-aws-lb-controller-irsa" # the name of the ALB IRSA
  }

  cluster_autoscaler_helm_config = {
    irsa_iam_role_name = "my-cluster-autoscaler-role-for-irsa" # the name of the Cluster autoscaler IRSA
  }

  tags = local.tags
}

You can also open an issue for adding the capability to provide the path and the name of the default policy created and attached to IRSA since it is a piece of puzzle which is missing here for your use-case.

[mikeinton]

@florentio - thanks for the info, I wasn't aware of that new issue/PR but it sounds like it will address exactly what we need here.

[florentio]

Awseome!!!

[github-actions[bot]]

This issue has been automatically marked as stale because it has been open 30 days with no activity. Remove stale label or comment or this issue will be closed in 10 days

[github-actions[bot]]

Issue closed due to inactivity.