EKS.AwsAuth is only accessible within its parent stack

[NinoSkopac]

Describe the feature

I cant access AwsAuth from anywhere but within its parent stack. I'm creating a State Machine in one stack and EKS Cluster in another.

If I try to import it into another stack using a public property in TS, I get a circular reference error:

const app = new cdk.App();
const eks = new EksClusterCore(app, 'Cluster', {});
const stRoleMapping = new StRoleMapping(app, 'RoleMapping', {
    awsAuth: eks.awsAuth
});

Gives: Error: StRole should be defined in the scope of the Cluster stack to prevent circular dependencies

If I import the EKS cluster, the AwsAuth class doesn't exist:

export class StRoleMapping extends cdk.Stack {
    constructor(scope: Construct, id: string, props?: StackProps) {
        super(scope, id, props);

        const cluster = aws_eks.Cluster.fromClusterAttributes(scope, 'Cluster', {
            clusterName: aws_ssm.StringParameter.fromStringParameterName(
                scope, 'ClusterName', 'ClusterName'
            ).stringValue,
            clusterEndpoint: aws_ssm.StringParameter.fromStringParameterName(
                scope, 'ClusterEndpoint', 'ClusterEndpoint'
            ).stringValue,
            clusterCertificateAuthorityData: aws_ssm.StringParameter.fromStringParameterName(
                scope, 'ClusterCertificateAuthorityData', 'ClusterCertificateAuthorityData'
            ).stringValue,
            kubectlRoleArn: aws_ssm.StringParameter.fromStringParameterName(
                scope, 'KubectlRoleArn', 'KubectlRoleArn'
            ).stringValue,
        });
        const stRoleArn = aws_ssm.StringParameter.fromStringParameterName(
            this, 'StateMachineRoleArn', 'StateMachineRoleArn'
        ).stringValue;

        cluster.awsAuth.addRoleMapping(stRoleArn, { // <== problem
          username: 'foobar',
          groups: []
        })
    }
}

Gives me Property 'awsAuth' does not exist on type 'ICluster'

I can import the cluster - I can write manifests to it - but it doesn't have the AwsAuth construct.

If I instantiate AwsAuth again, I lose connection to EKS because presumably the config-map/aws-auth gets overwritten, since AwsAuth is a construct.

Use Case

I want to have 1 EKS cluster and a lot of apps that use it. I want each app to be able to set its role mappings against EKS. Each app is a different stack.

Proposed Solution

Make sure AwsAuth is available for imported clusters.

Other Information

Since I can't do it in aforementioned ways, I'd probably do a strategic merge update on configmap/aws-auth:

• https://stackoverflow.com/questions/54571185/how-to-patch-a-configmap-in-kubernetes
• https://kubernetes.io/docs/tasks/manage-kubernetes-objects/update-api-object-kubectl-patch/#use-a-strategic-merge-patch-to-update-a-deployment

Acknowledgements

• [ ] I may be able to implement this feature request
• [ ] This feature might incur a breaking change

CDK version used

2.50.0 (build 4c11af6)

Environment details (OS name and version, etc.)

MacOS Ventura

[peterwoodworth]

To add a role mapping, the role must be guaranteed to already exist. I would recommend writing this code in such a way that guarantees the role exists at the time the Cluster stack is synthesized. You can accomplish this through importing a Role in the Cluster stack and adding the role mapping there.

We left some comments in the code explaining the decision to throw a circular dependency error: https://github.com/aws/aws-cdk/blob/4c11af6067b35125781aa590bb33c7990078d1ed/packages/%40aws-cdk/aws-eks/lib/aws-auth.ts#L106-L109

Additionally, Like you mentioned, AwsAuth is a construct and so it creates the underlying CloudFormation resources, meaning if you attempt to recreate this construct in another stack it will have unintended effects.

So, I don't think there's any particular feature we can implement to make this possible due to the limitations present making the functionality the way it is in the first place. I think it will need to be up to the user to be able to work around these limitations with custom resources, or the order in which resources are provisioned. Though if you think the restriction I linked above in our code is overly restrictive and prevents a use case, let us know

[github-actions[bot]]

This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.

[NinoSkopac]

Thank you for your response @peterwoodworth

The thing is i need to create the cluster first because the state machine makes api calls to it (the EKS).

My idea:

1. Create cluster
2. Create state machine that makes api calls to cluster
3. Add role mapping

I can't create the state machine first because the cluster doesn't exist yet. I can create cluster first, state machine second, but can't do the role mapping in the cluster stack because the state machine doesn't exist yet and i can't do it in a third stack because of circular reference.

I haven't been aware of custom resources, thanks for that. Would that be the way to go?

[mvs5465]

Though if you think the restriction I linked above in our code is overly restrictive and prevents a use case, let us know

@peterwoodworth Relying on lookups means that you cannot synthesize a stack unless you are currently authenticated to the target account. This prevents CI linting and test checks from working unless your CI runner is granted permissions to your AWS account. However, if you allow the user to reference a role from another stack then you can run a synth without authentication.

@NinoSkopac did you figure out a solution to this?

[NinoSkopac]

@mvs5465 nope

[mvs5465]

I think my point above is wrong in the sense that committing your cdk.context.json file is a better way to allow CI checks without needing to auth for lookups.