sops icon indicating copy to clipboard operation
sops copied to clipboard

SOPS Can't Find Data Key Required to Decrypt the SOPS File (AWS KMS)

Open better-sachin opened this issue 5 years ago • 4 comments

We are running into an issue on our Kubernetes pods (using kube2iam to provide IAM credentials to containers) trying to decrypt SOPS secrets where the assumed role tries to assume itself before decrypting.

Error message:

Failed to get the data key required to decrypt the SOPS file.
Group 0: FAILED
  arn:aws:kms:us-east-1:<account-id>:key/<uuid>: FAILED
    - | Error decrypting key: NoCredentialProviders: no valid
      | providers in chain. Deprecated.
      |     For verbose messaging see
      | aws.Config.CredentialsChainVerboseErrors
  
  arn:aws:kms:us-east-1:<account-id>:key/<uuid>: FAILED
    - | Error creating AWS session: Failed to assume role
      | "arn:aws:iam::<account-id>:role/service/<role-name>":
      | AccessDenied: User:
      | arn:aws:sts::<account-id>:assumed-role/<role-name>/<role-session-name>
      | is not authorized to perform: sts:AssumeRole on resource:
      | arn:aws:iam::<account-id>:role/service/<role-name>
      |     status code: 403, request id:
      | a401448b-6242-46d1-80d7-7e14396b4ad0
Recovery failed because no master key was able to decrypt the file. In
order for SOPS to recover the file, at least one key has to be successful,
but none were.https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_identifiers.html

— Is there a reason why SOPS tries to assume a role already assumed? — Is there a way to set the trust relationship of the role to be able to assume itself?

better-sachin avatar Sep 11 '20 15:09 better-sachin

Is there a reason why SOPS tries to assume a role already assumed?

Yes, the AWS master key you put in the SOPS file has a profile stored, and SOPS will try to assume it every time.

autrilla avatar Sep 14 '20 14:09 autrilla

looks like this is a known issue here: https://github.com/jtblin/kube2iam/issues/136

better-sachin avatar Sep 15 '20 15:09 better-sachin

Running into a similar issue: We have users which can not assume roles but already have the correct role and users, which have to assume a role. Because the role is assumed every time we can not use the KMS+Role syntax, as it will fail with half of our users.

Wouldn't it be possible/nice to check the current role before assuming it in https://github.com/mozilla/sops/blob/e1edc059487ddd14236dfe47267b05052f6c20b4/kms/keysource.go#L182 ?

Of course in most of the cases on can do the assume role manually before using SOPS, but there are some cases in which it does not work, like with the terraform sops plugin.

cpflaume avatar Sep 27 '22 05:09 cpflaume

This is a pretty big issue. It breaks AWS EC2 instance profiles, which are the de facto way to provide IAM permissions to servers on AWS.

The instance profile role is immediately assumed, and cannot assume itself.

JonathanLorimer avatar Dec 28 '23 20:12 JonathanLorimer