containers-roadmap icon indicating copy to clipboard operation
containers-roadmap copied to clipboard

[EKS] [eks-pod-identity] [bug]: Setting the STS Session name in eks-pod-identity-agent

Open taer opened this issue 1 month ago • 3 comments

I think this is probably more of a bug report, but I cannot find the proper channels. Version Info: EKS: 1.29 Eks Pod ID agent: v1.2.0-eksbuild.1

We are using eks-pod-identity. It's been working great till we started using IAM based kafka.

I directly hit the eks-pod-identity pod inside a container agent via

       AWS_CONTAINER_CREDENTIALS_FULL_URI:      http://169.254.170.23/v1/credentials                                                                                                        
       AWS_CONTAINER_AUTHORIZATION_TOKEN_FILE:  /var/run/secrets/pods.eks.amazonaws.com/serviceaccount/eks-pod-identity-token                                                               │

I took the resulting AccessKey, SecertKey, and Token, and used them to do a aws sts get-caller-identity

{
    "UserId": "AROAXYZP66II6MSBPLOUU:eks-k8s-wl-dev-engine-boo-5af5e7ac-5754-49ea-b28f-2c2a2eb95fbb",
    "Account": "BLAH",
    "Arn": "arn:aws:sts::BLAH:assumed-role/dev-use1-default-bookingApiPod-20240529171038526700000001/eks-k8s-wl-dev-engine-boo-5af5e7ac-5754-49ea-b28f-2c2a2eb95fbb"
}

The SessionName is non static, and I can't find any way to force set it. The issue comes from the MSK IAM usage. MSK doesn't allow "reauthentication". We get this error when the session name changes

failed authentication due to: Cannot change principals during re-authentication from IAM.arn:aws:sts::BLAH:assumed-role/prd-use1-default-bookingApiPod-20240529174141238800000002/eks-k8s-wl-prd-engine-boo-064f1ed1-2349-4774-b895-9a69ccc3eeb1: IAM.arn:aws:sts::BLAH:assumed-role/prd-use1-default-bookingApiPod-20240529174141238800000002/eks-k8s-wl-prd-engine-boo-875fdfcd-19cd-4ac3-8544-7077f94a6e39

Most services we've used IAM for to date don't care. The solution normally would be to set AWS_ROLE_SESSION_NAME when calling STS:AssumeRole. But we're not calling that, the eks-pod-identity pod is. The ContainerCredentialProvider in the SDK is just calling the AWS_CONTAINER_CREDENTIALS_FULL_URI URL with the contents of AWS_CONTAINER_AUTHORIZATION_TOKEN_FILE as the Authorization header. There is no option as far as I've found to have that include a constant session name to satisfy MSK.

Thanks!

taer avatar May 30 '24 17:05 taer