community
community copied to clipboard
aws-controllers-k8s
Ability to reference AWS_REGION, AWS_ACCOUNT_ID or CLUSTER_OIDC_PROVIDER_ID etc from within resources
Not sure how niche this scenario is, but here's two examples I've hit so far.
Example 1 - I'm creating a KMS Key. But one of the spec fields contains the AWS_ACCOUNT_ID. The controller obviously knows the AWS_ACCOUNT_ID, but I'd prefer my namespaced Key to not have to know it.
apiVersion: kms.services.k8s.aws/v1alpha1
kind: Key
metadata:
name: a-kms-key
spec:
policy: |
{
"Version": "2012-10-17",
"Id": "key-default-1",
"Statement": [
{
"Sid": "Enable IAM User Permissions",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::722198123436:root"
},
"Action": "kms:*",
"Resource": "*"
}
Example 2 - I'm creating an IAM Role, and the AWS_ACCOUNT_ID and CLUSTER_OIDC_PROVIDER_ID are both required to construct the policy document. Again, the controller already knows these values, but I'd prefer this namespaced resource not to.
apiVersion: iam.services.k8s.aws/v1alpha1
kind: Role
metadata:
name: some-iam-role
spec:
name: some-iam-role
assumeRolePolicyDocument: |
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Federated": "arn:aws:iam::722198123436:oidc-provider/CLUSTER_OIDC_PROVIDER_ID"
},
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition": {
"StringEquals": {
"CLUSTER_OIDC_PROVIDER_ID:sub": "system:serviceaccount:some-service-account"
}
}
}
]
}
policies:
- arn:aws:iam::722198123436:policy/some-iam-policy
Is there any concept of centrally stored variables that we can reference somehow? Or perhaps a set of ENVs that will always be find-replaced? Eg:
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::${AWS_ACCOUNT_ID}:root"
},
"Action": "kms:*",
"Resource": "*"
Hey @robertgates55 ! Thanks for this suggestion, I quite like this idea. Maybe my only concern with something like this is that it takes what are otherwise static manifests and makes them dynamic to the environment they're running in. This wouldn't necessarily be a problem, but it does make it more difficult to understand what is being declared. A similar problem already exists when we use resource references, but that is more explicit about entirely overriding a field value with another - whereas this is just a partial injection.
With that being said, I have actually had the idea of string interpolation/injection washing around in my head for a while, now. The IAM controller is exactly what sparked it for me too, where you need to build ARNs using string values from a few different resource field properties. The reason I haven't acted on this so far, though, is that it expands the scope of the ACK controllers from simply creating AWS resources based on what's in the cluster etcd into now templating and abstracting away inputs in K8s manifests. I am not sure if there are other K8s controllers that do something like that already, as an abstraction on top of manifests (maybe through annotations/other CRDs), but that would be preferable.
I also quite like this idea, as it reduces the chance of user input errors.
This would be great. We're having to use Helm templating to achieve this currently.
Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/aws-controllers-k8s/community.
/lifecycle stale
Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/aws-controllers-k8s/community.
/lifecycle rotten
Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Provide feedback via https://github.com/aws-controllers-k8s/community.
/close
@eks-bot: Closing this issue.
In response to this:
Rotten issues close after 30d of inactivity. Reopen the issue with
/reopen. Provide feedback via https://github.com/aws-controllers-k8s/community. /close
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.
@A-Hilaly: Reopened this issue.
In response to this:
/reopen /lifecycle frozen
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.
+1 on this. currently looking at storing these values in a secret, and using a helm lookup to template them.
Related: https://github.com/aws-controllers-k8s/community/issues/1694 If the policy was broken out as I describe there, then we likely wouldn't need templating, but just a special key to reference the current e.g. OIDC. i.e. instead of
principal:
type: federated
value: arn:aws:iam::123456789:oidc-provider/myoidcprovider.example.com
we could add a feature to have:
principal:
type: federated
valueFrom:
operatorDefault: true
Though it's very tempting, I think it's best to avoid templating in the general case and prefer "kubernetes-native" fields.
