fully private cluster with lambda or dedicated bastion host as a proxy

[tomiszili]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

What is the outcome that you are trying to reach?

I want to create and manage a fully private EKS clusters without any additional VPC, VPC peering, Cloud9 instance etc.... The idea is came from the aws-quickstart-eks cloudformation templates. Currently if you want to provision/modify anything inside the eks cluster - without the Cloud9 instance - (e.g.: edit the aws-auth configmap, deploying addons with helm charts or any kubernetes manifest...) you have to enable the public endpoint, then take the desired actions and then you have to disable the public endpoint again. As far as i know Terraform could not handle changes delayed in time to the same cluster, and consider the latest sdk action as the desired state, so this public endpoint on-off switching is unbearable.

Describe the solution you would like

One possible solution could be if the aws-auth, helm, and kubernetes providers could communicate through a lambda proxy attached to the same subnets as the EKS cluster. I know about the 15 minutes limitation (maybe step functions orchestrated lambdas?). I think the 15 minutes upper limit couldn't be a problem because in most cases only management addons (logging, monitoring, security, etc...) deployed with terraform.

Other possibility is to create a dedicated bastion host (in private subnet) to every cluster (e.g.: t4g.micro ec2) with ssm-agent installed due to open a ssm session with port forwarding (e.g.: for SSH tunneling purposes) to the EKS cluster's https endpoint. This tunneling acts as a proxy for the terraform providers.

Describe alternatives you have considered

Additional context

I'm not a terraform guru so i don't know if my idea could be implemented at all. Hope for the best.