Authenticating into EKS cluster thru OIDC creates an error with kubent

[kjenney]

We're trying to integrate kubent into our automation workflows. We're running GitLab with self-hosted runners that are deployed to an EKS cluster. When we run GItLab jobs within our pipelines the Kubernetes executor pods are using IRSA with OIDC to assume an IAM role into clusters that we manage. We can run kubectl commands without any issue, however when we run kubent it's giving us the following error:

ERR Failed to retrieve data from collector error="list: failed to list: secrets is forbidden: User \"system:serviceaccount:redacted:atlas-redacted-executor\" cannot list resource \"secrets\" in API group \"\" at the cluster scope" name="Helm v3"

NOTE: The serviceaccount name has been redacted for security purposes. When I run kubent with a log-level of debug I also see this same error for every other type of resource - which makes sense seeing as the serviceaccount doesn't have access to this cluster.

The issue is that the identity that we are using to connect to the cluster is not the serviceaccount, but the assumed role.

$kubectl auth whoami | grep Username
Username    admin
Extra: arn                                             [arn:aws:sts::redacted:assumed-role/redacted-iam-role/AWSCLI-Session]

How can we use kubent to autenticate with EKS clusters from a pod running in another EKS cluster?

[mcmarkj]

Facing this exact same issue but on GKE.

ERR Failed to retrieve data from collector error="list: failed to list: secrets is forbidden: User \"system:serviceaccount:github-actions:gha-rs-no-permission\" cannot list resource \"secrets\" in API group \"\" at the cluster scope" name="Helm v3"

Whereas it has it's own service account with the right permissions:

TTRIBUTE                                VALUE
Username                                 terraform-github-actions@<redacted>.iam.gserviceaccount.com