sops icon indicating copy to clipboard operation
sops copied to clipboard

Error decrypting key: AccessDeniedException with AWS KMS

Open alexandr-x-ursul opened this issue 2 years ago • 0 comments

Hello,

I've encountered an issue/QoL deficit which I haven't been able to find among other issues.

When I encrypt something as follows:

sops --aws-profile <profile> --kms <key arn> -e --output-type yaml <secret file> > secrets.encrypted.yaml

As part of the output, the following is generated:

sops:
    kms:
        - arn: arn:aws:kms:xxx
          created_at: "2022-06-08T14:21:13Z"
          enc: xxx
          aws_profile: xxx

When running this with FluxCD, and service account has permissions with an annotation to a role, it fails with an error similar to:

Error decrypting key: AccessDeniedException: The ciphertext
      | refers to a customer master key that does not exist, does
      | not exist in this region, or you are not allowed to access.
      | 	status code: 400

despite permissions being otherwise in order.

It appears to me that sops attempts to use the aws_profile used during encryption as generated at the encryption time. Simply removing this part of the generated encrypted file will allow sops running inside of FluxCD to decrypt objects successfully.

This is also verifiable on CLI. Given no configured profiles, attempting to decrypt a file which has a sops.kms configuration with a specified aws_profile will fail with a similar error as above.

The QoL part of the issue is that encrypting a file with an aws_profile will require further manual edits on the encrypted file to ensure it can be consumed by Flux, also decrypting the file on CLI with:

sops --aws-profile <profile> --kms <key id> -d secrets.encrypted.yaml will not work unless aws_profile is specified on the file as well, even though we are passing the --aws-profile on CLI.

In summary, one has to be very careful with encrypting/decrypting files with sops if they are using an aws profile and have to manually modify the files after the fact for FuxCD consumption. This issue has personally cost me hours of debugging before I realized why sops on Flux was failing to decrypt my secrets.

My suggestion would be to not assume that the entity encrypting and the entity decrypting have equal credential configuration. I think SOPS should drop the aws_profile parameter it bakes into encrypted files and instead use default credentials, default should still be overridable by .sops config file.

This means that if you encrypt a file using an aws profile, and attempt to decrypt it without using one, it should be down to your default credentials / .sops config If you attempt to decrypt it while specifying --aws-profile then it should take that with highest priority.

Let me know if you need any further clarification. Cheers!

alexandr-x-ursul avatar Jun 08 '22 14:06 alexandr-x-ursul