AWS Profiles not properly resolving the .aws/config file

[gl-ahiggins]

AWS Profile not correctly resolving .aws/config file

When attempting to use a profile in a .aws/config file, sops does not properly resolve the credentials. Example sops and .aws config files are provided below. I believe I have a fix working, with a PR coming shortly.

SOPS file

hello: ENC[...]
creds:
    user: ENC[...]
    password: ENC[...]
sops:
    kms:
    -   arn: arn:aws:kms:us-east-1:<account_id>:key/<key_id>
        created_at: '2020-06-09T16:55:32Z'
        enc: "..."
        aws_profile: my-account # This profile exists in .aws/config
    gcp_kms: []
    azure_kv: []
    lastmodified: '2020-06-09T16:57:40Z'
    mac: "..."
    pgp: []
    unencrypted_suffix: _unencrypted
    version: 3.5.0

.aws/config file

Here we're defining a profile where we assume a role into a separate account. While assuming roles is possible with sops using the kms_arn+role_arn configuration, our organization uses dozens of roles to manage AWS access. Different teams accessing the same KMS key are likely to have different roles. Ideally this is solved through normal behavior of the AWS sdk, rather than be reliant on sops' implementation.

[profile my-account]
source_profile = my-main-account
role_arn = arn:aws:iam::<account_id>:role/my-automation-role

Existing problems

SOPS does not currently resolve the profile correctly. This prevents standard usage where profiles defined in the .aws/config file assume AWS credentials before executing commands. The following command works with the aws cli, setting a profile through the environment variables:

# Properly assumes role through .aws/config and prints buckets
AWS_PROFILE=my-account aws s3 ls

In sops, the following is returned:

AWS_PROFILE=my-accounts sops my-file.yaml
# ...
Group 0: FAILED
  arn:aws:kms:us-east-2:<account_id>:key/<key_id>: FAILED
    - | Error decrypting key: SharedCredsLoad: failed to get profile

Recovery failed because no master key was able to decrypt the file. In
order for SOPS to recover the file, at least one key has to be successful,
but none were.

Similarly, if no AWS_PROFILE is provided, the one from the sops file is used. This similarly fails to resolve:

sops my-file.yml
# ...
Failed to get the data key required to decrypt the SOPS file.

Group 0: FAILED
  arn:aws:kms:us-east-2:668652621010:key/c24e651c-578c-4602-ab4b-30d85770d963: FAILED
    - | Error decrypting key: NoCredentialProviders: no valid
      | providers in chain. Deprecated.
      |         For verbose messaging see
      | aws.Config.CredentialsChainVerboseErrors

Recovery failed because no master key was able to decrypt the file. In
order for SOPS to recover the file, at least one key has to be successful,
but none were.

Cause

The above problems seem to be caused by improperly instantiating an AWS session:

func (key MasterKey) createSession() (*session.Session, error) {
	re := regexp.MustCompile(`^arn:aws[\w-]*:kms:(.+):[0-9]+:(key|alias)/.+$`)
	matches := re.FindStringSubmatch(key.Arn)
	if matches == nil {
		return nil, fmt.Errorf("No valid ARN found in %q", key.Arn)
	}

	config := aws.Config{Region: aws.String(matches[1])}

    // Does not respect the .aws/config file. Forces the profile to be in .aws/credentials, preventing standard assume roles though the ./aws/config file.
	if key.AwsProfile != "" {
		config.Credentials = credentials.NewSharedCredentials("", key.AwsProfile)
	}

    // Not providing a SharedConfigState option makes .aws/config ignored by default
    // This can be manually set with AWS_SDK_LOAD_CONFIG, but most CLI tools have this enabled by default to just work
    // Can also pass the profile in here, which respects the .aws/config file
	opts := session.Options{
		Config:                  config,
		AssumeRoleTokenProvider: stscreds.StdinTokenProvider,
	}
	sess, err := session.NewSessionWithOptions(opts)
	if err != nil {
		return nil, err
	}
	if key.Role != "" {
		return key.createStsSession(config, sess)
	}
	return sess, nil
}

[sc250024]

Same thing. I'm using Sops v3.6.0, with credentials obtained via gimme-aws-creds through Okta. This means that the ~/.aws/credentials file uses temporary aws_access_key_id and aws_secret_access_key keys.

Creating a new file results in the following error:

$ AWS_PROFILE=development sops test.yaml

Results in the following error:

$ sops --verbose test.yaml

[AWSKMS]     INFO[0005] Encryption failed                             arn="arn:aws:kms:us-east-1:111122223333:key/11111c1d-b2c8-437d-ae4b-d00a123ccc45"
Error encrypting the data key with one or more master keys: [failed to encrypt new data key with master key "arn:aws:kms:us-east-1:111122223333:key/11111c1d-b2c8-437d-ae4b-d00a123ccc45": Failed to call KMS encryption service: NoCredentialProviders: no valid providers in chain. Deprecated.
    For verbose messaging see aws.Config.CredentialsChainVerboseErrors]

But I can use the key just fine with the AWS cli:

$ AWS_PROFILE=development aws kms encrypt \
--key-id 11111c1d-b2c8-437d-ae4b-d00a123ccc45 \
--plaintext my-secret-text \
--query CiphertextBlob \
--output text

So yeah.... a bit annoying.

[abeluck]

This is also affecting us using sops 3.6.1

AWS_SDK_LOAD_CONFIG=1 AWS_PROFILE=development ./sops-v3.6.1.linux --verbose test.sops.yml
[AWSKMS]	 INFO[0001] Encryption failed                             arn="arn:aws:kms:eu-west-1:694143964892:key/840b9097-bcbc-49c4-8bb8-e5cdf2a6234c"
Error encrypting the data key with one or more master keys: [failed to encrypt new data key with master key "arn:aws:kms:eu-west-1:694143964892:key/840b9097-bcbc-49c4-8bb8-e5cdf2a6234c": Failed to call KMS encryption service: NoCredentialProviders: no valid providers in chain. Deprecated.
	For verbose messaging see aws.Config.CredentialsChainVerboseErrors]

.sops.yaml

---
creation_rules:
  - key_groups:
    - kms:
      - arn: arn:aws:kms:eu-west-1:xxx:key/xxxx

[ashokpokharel977]

AWS Profiles are respected when using outside of a key group but using with a key group does not respect the profile option.

creation_rules:
  - path_regex: \.dev\.yaml$
    key_groups:
      - kms:
          - arn: role_arn
            aws_profile: dev

[jseiser]

Same issue here.

We use AWS SSO through Okta. Our users are in 1 of 2 roles, that have KMS access.

sops:
    kms:
    -   arn: arn:aws:kms:us-east-1:xxxxxx:key/b2ad0cb6-key-here-3ef25dc242bf
        created_at: '2019-09-12T15:45:08Z'
        enc: '...'
        aws_profile: ""

This results in them having a profile, with access to the key, but sops fails to decrypt.

❯ AWS_PROFILE="developer" sops --verbose --decrypt minikube/secrets2.yaml
[AWSKMS]	 INFO[0005] Decryption failed                             arn="arn:aws:kms:us-east-1:xxxxxx:key/b2ad0cb6-key-here-3ef25dc242bf"
Failed to get the data key required to decrypt the SOPS file.

Group 0: FAILED
  arn:aws:kms:us-east-1:xxxxxx:key/b2ad0cb6-key-here-3ef25dc242bf: FAILED
    - | Error decrypting key: NoCredentialProviders: no valid
      | providers in chain. Deprecated.
      | 	For verbose messaging see
      | aws.Config.CredentialsChainVerboseErrors

Recovery failed because no master key was able to decrypt the file. In
order for SOPS to recover the file, at least one key has to be successful,
but none were.

[MasatoshiMizumoto]

I am also encountering this event with 3.7.1. My encrypted files were stored in json, so I got around it by deleting the aws_profile using jq.

cat myfile.enc | jq '. | del(.sops.kms[0].aws_profile)' > myfile.temp

[nmguse-bighealth]

One can work around this issue by creating a shell alias that wraps sops and injects temporary credentials.

sops() {
    session=$(aws sts assume-role --profile $(aws configure get source_profile) --role-arn $(aws configure get role_arn) --role-session-name "${USER}-sops")

    access_key_id=$(echo ${session} | jq '.Credentials.AccessKeyId' -r)
    secret_access_key=$(echo ${session} | jq '.Credentials.SecretAccessKey' -r)
    session_token=$(echo ${session} | jq '.Credentials.SessionToken' -r)

    AWS_ACCESS_KEY_ID=${access_key_id} AWS_SECRET_ACCESS_KEY=${secret_access_key} AWS_SESSION_TOKEN=${session_token} command sops $@
}

[abeluck]

AWS Profiles are respected when using outside of a key group but using with a key group does not respect the profile option.

This is still an issue, and quite a problematic one. Now that sops is getting some love again from the community would it be possible for someone with the Go dev knowledge to troubleshoot? This issue.