aws
aws-eks: KubernetesManifest "replace" option
Describe the feature
I would like to have a "replace" option for the aws-cdk.aws-eks.KubernetesManifest construct. This should first delete the old Manifest and then apply the new one.
Use Case
Some Kubernetes resources are immutable. StorageClasses for instance. If you do not delete those resources by hand before the next cdk deployment, the deployment will be rollback with an error. That is because Kubernetes reports an error back to CloudFormation.
It is hard to figure out, which immutable K8s resources might be affected by an update. Especially if you have many of them and are not the creator of the stack.
When the stack update also affects resources like RDS or ActiveMQ an update/rollback of the complete stack might take hours.
That's why the safest solution would be, marking some KubernetesManifest constructs with a replace: true flag and further avoid any problems with them.
Proposed Solution
No response
Other Information
No response
Acknowledgements
- [ ] I may be able to implement this feature request
- [ ] This feature might incur a breaking change
CDK version used
1.187.0
Environment details (OS name and version, etc.)
independent
I am very interested in this case. Can you provide a minimal CDK app that I can reproduce it in my account?
Also, there's actually a prune option: https://github.com/aws/aws-cdk/blob/44a4812778d87af27809e5a733c6e5ea6b65004b/packages/%40aws-cdk/aws-eks/lib/kubectl-handler/apply/init.py#L65-L67 https://github.com/aws/aws-cdk/blob/44a4812778d87af27809e5a733c6e5ea6b65004b/packages/%40aws-cdk/aws-eks/lib/k8s-manifest.ts#L36
Will it be helpful by turning on this option?
Hi @pahud
Can you provide a minimal CDK app that I can reproduce it in my account?
Of course. This minmal example (written in Python) should allow to experiment with. After the deployment you can change the ConfigMaps value for instance. When tracing the ConfigMap in Kubernetes, it should just be updated but not deleted. The creation timestamp should remain the same.
from aws_cdk import App, Stack
from aws_cdk.aws_eks import Cluster, KubernetesVersion, KubernetesManifest
app = App()
stack = Stack(app, "stack")
cluster = Cluster(stack, "cluster", version=KubernetesVersion.V1_21)
KubernetesManifest(
stack,
"manifest",
cluster=cluster,
overwrite=True,
prune=True,
manifest=[
{
"apiVersion": "v1",
"kind": "ConfigMap",
"metadata": {
"name": "demo",
},
"data": {
"foo": "bar",
},
}
],
)
app.synth()
Will it be helpful by turning on this [prune] option?
No, unfortunately not. It will just delete the Manifest in Kubernetes in the case it is removed from the CDK/CFN deployment.
When this option is enabled (default), the construct will inject a label to all Kubernetes resources included in this manifest which will be used to prune resources when the manifest changes via kubectl apply --prune.
Source: https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_eks.KubernetesManifest.html#prune
