ex_aws and ex_aws_sts does not seems to support IAM authentication within EKS?

[danibachar]

• Do not use the issues tracker for help or support (try Elixir Forum, Slack, IRC, etc.)
• Questions about how to contribute are fine.

Environment

• Elixir & Erlang versions (elixir --version):1.16
• ExAws version mix deps |grep ex_aws: 2.5.2 (ex_aws_s3: 2.5.2, ex_aws_sts: latest)
• HTTP client version. IE for hackney do mix deps | grep hackney

Current behavior

I have an EKS cluster, when deploying a Deployment using a docker file. The SDK seems to fail to authenticate with the the ServiceAccount that is attached to that deployment. It seems to default to an instance_role and auth with an IAM role of the nodes in the cluster. In our example this IAM role does not have permissions to operate with an S3 bucket. Only the IAM role we have configure with the ServiceAccount.

I have checked that the newly create IAM role has succfient permissions and can operate with the relevant S3

Expected behavior

Working within EKS the SDK should work like any other AWS SDK and allow assuming/working with the IAM role that is attached to a Pod/Deployment using a ServiceAccount

[pepicrft]

👋🏼 We are interested in contributing to this. Could you let us know if this aligns with the library's direction and perhaps provide some pointers around how you'd expect it to be implemented?

[danibachar]

Thanks Pedro! the containers in your Pods must use an AWS SDK version that supports assuming an IAM role through an OpenID Connect web identity token file.

https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts-minimum-sdk.html

[arifken]

To add to what to @danibachar, in order for this sdk to be able to run properly in EKS using service accounts, I think you'll need to implement something like this in the default credential provider chain for this SDK.

[RobinFrcd]

Hi, Is there any known workaround right now ? Giving permissions to the node (instance) directly is clearly a very bad practice, and I would like to avoid that as much as possible.

Thanks !

[amacciola]

Hello, We found this issue in our production EKS clusters today. We had not noticed the issue for some time because most of our applications used the same S3 permissions. Which we had not to our knowledge had as apart of our Nodes instance_roles permissions. (which we need to now remove)

When we spun up a new application that needed a new service_account, with only AWS Bedrock access. We found this issue. Since it was not applying the pod level service account, only giving it access to the Nodes instance_role.

This is a major issue for this library trying to run in EKS using service accounts.

Any progress made towards a fix on this would be greatly appreciated.

[ojundt]

I was also just battling this and this seems to work:

config :ex_aws,
  access_key_id: [{:awscli, "default", 30}],
  secret_access_key: [{:awscli, "default", 30}],
  awscli_auth_adapter: ExAws.STS.AuthCache.AssumeRoleWebIdentityAdapter,
  awscli_credentials: %{
    "default" => %{}
  }

Requires https://github.com/ex-aws/ex_aws_sts