cluster-api-provider-aws Load balancer is created but EC2 instances are not created

Load balancer is created but EC2 instances are not created

Open oneironautjack opened this issue 10 months ago • 12 comments

trafficstars

/kind bug

What steps did you take and what happened: When deploying CAPI workload cluster with existing VPC, the classic load balancer is created but the EC2 instances are not created. The Machine object is created for the first control plane node but the corresponding instance is not. The Machine object is stuck in Pending. I've tried adding the skip preflight annotation as well.

What did you expect to happen: After the load balancer is created, the instances for the control plane should spawn and cloud init should bootstrap Kubeadm.

Anything else you would like to add: [Miscellaneous information that will assist in solving the issue.] The below error message is logged so I expect the instance should be created. I tried looking for the request via CloudTrail but there doesn't seem to be an event for RunInstances.

 Waiting for control plane to pass preflight checks to continue reconciliation: Machine my-control-plane-rdqt6 does not have a corresponding Node yet (Machine.status.nodeRef not set)

Cluster/beta-workload-cluster                                                        False  Warning   ScalingUp                        153m   Scaling up control plane to 3 replicas (actual 1)

├─ClusterInfrastructure - AWSCluster/beta-aws-workload-cluster                       True                                              153m                                   

├─ControlPlane - KubeadmControlPlane/beta-control-plane                              False  Warning   ScalingUp                        153m   Scaling up control plane to 3 replicas (actual 1)

│ └─Machine/beta-control-plane-rdqt6                                                 False  Info      WaitingForBootstrapData          153m   0 of 2 completed                

│   └─BootstrapConfig - KubeadmConfig/beta-control-plane-rdqt6                       False  Info      WaitingForControlPlaneAvailable  153m                                   

└─Workers                                                                                                                                                                     

  ├─MachineDeployment/beta-dbbases-deployment                                      False  Warning   WaitingForAvailableMachines      153m   Minimum availability requires 1 replicas, current 0 available

  │ └─Machine/beta-dbbases-deployment-qcn7z-w24f7                                  False  Info      WaitingForBootstrapData          153m   0 of 2 completed                  
 


│   └─BootstrapConfig - KubeadmConfig/beta-dbbases-deployment-qcn7z-w24f7        False  Info      WaitingForControlPlaneAvailable  153m                                        

  ├─MachineDeployment/beta-dbservices-deployment                                   False  Warning   WaitingForAvailableMachines      153m   Minimum availability requires 1 replicas, current 0 available     

  │ └─Machine/beta-dbservices-deployment-klk4j-hdsz8                               False  Info      WaitingForBootstrapData          153m   0 of 2 completed                                

  │   └─BootstrapConfig - KubeadmConfig/beta-dbservices-deployment-klk4j-hdsz8     False  Info      WaitingForControlPlaneAvailable  153m                                                                     

  ├─MachineDeployment/beta-labservices-deployment                                        False  Warning   WaitingForAvailableMachines      153m   Minimum availability requires 1 replicas, current 0 available

  │ └─Machine/beta-labservices-deployment-59srj-8frgm                                    False  Info      WaitingForBootstrapData          153m   0 of 2 completed                                            

  │   └─BootstrapConfig - KubeadmConfig/beta-labservices-deployment-59srj-8frgm          False  Info      WaitingForControlPlaneAvailable  153m                                                               

  ├─MachineDeployment/beta-performance-gpu-deployment                                False  Warning   WaitingForAvailableMachines      153m   Minimum availability requires 1 replicas, current 0 available 

  │ └─Machine/beta-performance-gpu-deployment-s49wz-glzdj                            False  Info      WaitingForBootstrapData          153m   0 of 2 completed                                                

  │   └─BootstrapConfig - KubeadmConfig/beta-performance-gpu-deployment-s49wz-glzdj  False  Info      WaitingForControlPlaneAvailable  153m                                                     

  └─MachineDeployment/beta-worker-deployment                                         False  Warning   WaitingForAvailableMachines      153m   Minimum availability requires 1 replicas, current 0 available

Environment:

cluster-api-provider-hetzner version: clusterctl: v.1.84 clusterapi version v1.9.3
Kubernetes version: (use kubectl version) Client Version: v1.31.1 Kustomize Version: v5.4.2 Server Version: v1.30.5
OS (e.g. from /etc/os-release): RHEL 8.10

Jan 14 '25 20:01 oneironautjack

This issue is currently awaiting triage.

If CAPA/CAPI contributors determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Jan 14 '25 20:01 k8s-ci-robot

I do not think this is enough information to determine a cause. Can you provide more resources, e.g. Machine, AWSMachine? The CAPA controller logs would also help. Please redact them as necessary.

Jan 15 '25 00:01 dlipovetsky

I0115 14:25:26.740446 I0115 14:25:26.751807 I0115 14:25:26.751887 I0115 14:25:26.752038 I0115 14:25:26.752306 I0115 14:25:26.752523 I0115 14:25:26.752534 I0115 14:25:26.753059 I0115 14:25:26.754524 I0115 14:25:26.756087 I0115 14:25:26.758011 I0115 14:25:26.758450 I0115 14:25:26.758841 I0115 14:25:26.759398 I0115 14:25:26.759881 I0115 14:25:26.759991 I0115 14:25:26.760053 I0115 14:25:26.760139 I0115 14:25:26.760203 I0115 14:25:26.760256 I0115 14:25:26.760343 I0115 14:25:26.760390 I0115 14:25:26.760477 I0115 14:25:26.760540 I0115 14:25:26.760585 I0115 14:25:26.760660 I0115 14:25:26.760696 I0115 14:25:26.760748 I0115 14:25:26.760832 I0115 14:25:26.760881 I0115 14:25:26.760927 I0115 14:25:26.760993 I0115 14:25:26.761029 I0115 14:25:26.761075 I0115 14:25:26.761146 I0115 14:25:26.761213 I0115 14:25:26.761264 I0115 14:25:26.761309 I0115 14:25:26.761485 I0115 14:25:26.761550 I0115 14:25:26.761607 I0115 14:25:26.761707 I0115 14:25:26.761811 I0115 14:25:26.761832 I0115 14:25:26.761904 I0115 14:25:26.762553 I0115 14:25:26.763528 I0115 14:25:26.763892 I0115 14:25:26.763969 I0115 14:25:26.765258 I0115 14:25:26.865629 I0115 14:25:27.268901 2025/01/15 14:25:43 I0115 14:25:43.714211 I0115 14:25:43.714688 I0115 14:25:43.714721 I0115 14:25:43.714734 I0115 14:25:43.714743 I0115 14:25:43.714751 I0115 14:25:43.715188 I0115 14:25:43.715209 I0115 14:25:43.715218 I0115 14:25:43.715206 I0115 14:25:43.715239 I0115 14:25:43.715258 I0115 14:25:43.715264 I0115 14:25:43.715277 I0115 14:25:43.715288 I0115 14:25:43.715305 I0115 14:25:43.715311 I0115 14:25:43.715327 I0115 14:25:43.715339 I0115 14:25:43.715349 I0115 14:25:43.715376 I0115 14:25:43.715386 I0115 14:25:43.715392 I0115 14:25:43.715398 I0115 14:25:43.822218 I0115 14:25:43.825524 I0115 14:25:43.825557 I0115 14:25:43.825572 I0115 14:25:43.825526 I0115 14:25:43.825616 I0115 14:25:43.927218 I0115 14:25:43.979779 I0115 14:25:44.152841 I0115 14:25:45.579655 I0115 14:25:45.640210 I0115 14:25:45.692187 I0115 14:25:46.442344 I0115 14:25:46.492420 I0115 14:25:47.086512 I0115 14:25:47.139517 I0115 14:26:31.447122 I0115 14:26:31.510125 I0115 14:26:31.536734 I0115 14:26:31.550157 I0115 14:26:31.808391 I0115 14:26:31.826653 I0115 14:26:31.833588 I0115 14:26:31.839951 I0115 14:26:31.853531 I0115 14:26:31.854771 I0115 14:26:31.896571 I0115 14:26:31.924536 I0115 14:26:43.617976 I0115 14:26:43.630943 I0115 14:26:43.639227 I0115 14:26:43.674966 I0115 14:26:43.683337 I0115 14:26:43.702688 I0115 14:26:43.705806 I0115 14:26:43.710556 I0115 14:26:43.715840 I0115 14:26:43.730895 2025/01/15 14:26:43 I0115 14:26:43.912631 I0115 14:26:43.938171 I0115 14:26:43.943800 I0115 14:26:43.943857 I0115 14:26:43.943941 I0115 14:26:43.980511 I0115 14:26:43.985927 I0115 14:26:43.997217 I0115 14:26:43.997218 I0115 14:26:44.014531 I0115 14:26:44.018703 I0115 14:26:44.029568 I0115 14:26:44.043133 I0115 14:26:44.043862 I0115 14:26:44.065170 I0115 14:26:44.089280 I0115 14:26:44.135007 I0115 14:26:44.146182 I0115 14:26:44.148960 I0115 14:26:44.185634 I0115 14:26:44.189188 I0115 14:26:44.220208 I0115 14:26:44.236128 I0115 14:26:44.237990 I0115 14:26:44.256751 I0115 14:26:44.258920 I0115 14:26:44.274281 I0115 14:26:44.288205 I0115 14:26:44.301101 I0115 14:26:44.317234 I0115 14:26:44.322285 I0115 14:26:44.323199 I0115 14:26:44.343800 I0115 14:26:44.345761 I0115 14:26:44.384768 I0115 14:26:44.453524 I0115 14:26:44.537397 I0115 14:26:44.685995 I0115 14:26:44.724421 I0115 14:26:44.727340 I0115 14:26:44.761442 I0115 14:44:10.323790 I0115 14:44:10.324823 I0115 14:44:10.324954 I0115 14:44:10.326144 I0115 14:44:10.326200 I0115 14:44:10.330105 I0115 14:44:10.330184 I0115 14:44:10.564065 I0115 14:44:44.558207 I0115 14:44:44.558257 I0115 14:44:44.558802 I0115 14:44:44.558257 I0115 14:44:44.560006 I0115 14:44:44.560525 I0115 14:47:41.156137 I0115 14:47:41.156211 I0115 14:47:41.156137 I0115 14:47:41.156211 I0115 14:47:41.156748 I0115 14:47:41.156782 I0115 14:53:23.616955 I0115 14:53:23.617072 I0115 14:53:23.617159 I0115 14:53:23.617965 I0115 14:53:23.617159 I0115 14:53:23.618399 I0115 14:53:23.618265 I0115 14:53:23.729284 I0115 14:54:23.453189 I0115 14:54:23.453191 I0115 14:54:23.453852 I0115 14:54:23.453191 I0115 14:54:23.453189 I0115 14:54:23.453863 I0115 14:58:39.872045 I0115 14:58:39.872055 I0115 14:58:39.872616 I0115 14:58:39.872061 I0115 14:58:39.872061 I0115 14:58:39.872827 I0115 15:02:36.910748 I0115 15:02:36.911361 I0115 15:02:36.911372 I0115 15:02:36.911376 I0115 15:02:36.912408 I0115 15:02:36.912845 I0115 15:02:36.915322 I0115 15:02:37.011853 I0115 15:04:02.348654 I0115 15:04:02.348659 I0115 15:04:02.348665 I0115 15:04:02.349244 I0115 15:04:02.349249 I0115 15:04:02.348665 2025/01/15 15:06:59 I0115 15:09:38.588101 I0115 15:09:38.588756 I0115 15:09:38.589266 I0115 15:09:38.588101 I0115 15:09:38.590549 I0115 15:09:38.590572 I0115 15:11:50.204027 I0115 15:11:50.204223 I0115 15:11:50.204286 I0115 15:11:50.204848 I0115 15:11:50.204227 I0115 15:11:50.205301 I0115 15:11:50.205662 I0115 15:11:50.318549 I0115 15:13:41.244012 I0115 15:13:41.244313 I0115 15:13:41.244347 I0115 15:13:41.244610 I0115 15:13:41.244012 I0115 15:13:41.245177 I0115 15:20:37.303773 I0115 15:20:37.303822 I0115 15:20:37.303774 I0115 15:20:37.303785 I0115 15:20:37.304511 I0115 15:20:37.304805 I0115 15:21:03.497103 I0115 15:21:03.497229 I0115 15:21:03.497264 I0115 15:21:03.498154 I0115 15:21:03.498275 I0115 15:21:03.498646 I0115 15:21:03.499413 I0115 15:21:03.583008 I0115 15:23:20.139171 I0115 15:23:20.139191 I0115 15:23:20.139171 I0115 15:23:20.139171 I0115 15:23:20.139876 I0115 15:23:20.140036 I0115 15:30:16.790850 I0115 15:30:16.790952 I0115 15:30:16.791237 I0115 15:30:16.791237 I0115 15:30:16.791893 I0115 15:30:16.791909 I0115 15:30:16.792474 I0115 15:30:16.886231 I0115 15:31:36.019912 I0115 15:31:36.019933 I0115 15:31:36.019933 I0115 15:31:36.019919 I0115 15:31:36.020597 I0115 15:31:36.021276 I0115 15:32:59.034443 I0115 15:32:59.034536 I0115 15:32:59.034569 I0115 15:32:59.034519 I0115 15:32:59.035073 I0115 15:32:59.035755 I0115 15:39:30.084317 I0115 15:39:30.084501 I0115 15:39:30.084536 I0115 15:39:30.084592 I0115 15:39:30.085578 I0115 15:39:30.085602 I0115 15:39:30.085981 I0115 15:39:30.197962 I0115 15:42:34.736217 I0115 15:42:34.736218 I0115 15:42:34.736218 I0115 15:42:34.736722 I0115 15:42:34.736850 I0115 15:42:34.736218 I0115 15:42:37.929869 I0115 15:42:37.929872 I0115 15:42:37.930511 I0115 15:42:37.929882 I0115 15:42:37.929882 I0115 15:42:37.930930 2025/01/15 15:47:00 2025/01/15 15:47:00 I0115 15:48:43.377749 I0115 15:48:43.377958 I0115 15:48:43.378024 I0115 15:48:43.378042 I0115 15:48:43.378864 I0115 15:48:43.379162 I0115 15:48:43.379172 I0115 15:48:43.492321 I0115 15:52:16.825014 I0115 15:52:16.825035 I0115 15:52:16.825038 I0115 15:52:16.825107 I0115 15:52:16.826340 I0115 15:52:16.826347 I0115 15:53:33.452649 I0115 15:53:33.452877 I0115 15:53:33.452649 I0115 15:53:33.453217 I0115 15:53:33.452877 I0115 15:53:33.454035 I0115 15:57:56.672032 I0115 15:57:56.672160 I0115 15:57:56.672284 I0115 15:57:56.672895 I0115 15:57:56.673387 I0115 15:57:56.673614 I0115 15:57:56.673845 I0115 15:57:56.774843 I0115 16:01:55.720647 I0115 16:01:55.720970 I0115 16:01:55.721098 I0115 16:01:55.720657 I0115 16:01:55.721428 I0115 16:01:55.722207 I0115 16:04:32.168419 I0115 16:04:32.168485 I0115 16:04:32.168497 I0115 16:04:32.168419 I0115 16:04:32.169003 I0115 16:04:32.169830 2025/01/15 16:07:01 I0115 16:07:09.964946 I0115 16:07:09.965045 I0115 16:07:09.965119 I0115 16:07:09.965443 I0115 16:07:09.966129 I0115 16:07:09.966208 I0115 16:07:09.966387 I0115 16:07:10.093924 I0115 16:11:34.616002 I0115 16:11:34.616002 I0115 16:11:34.616014 I0115 16:11:34.616643 I0115 16:11:34.616691 I0115 16:11:34.616014 I0115 16:15:30.885520 I0115 16:15:30.885529 I0115 16:15:30.886129 I0115 16:15:30.886966 I0115 16:15:30.887134 I0115 16:15:30.888126 I0115 16:16:23.258256 I0115 16:16:23.258544 I0115 16:16:23.258602 I0115 16:16:23.258622 I0115 16:16:23.259561 I0115 16:16:23.259597 I0115 16:16:23.259669 I0115 16:16:23.369241 I0115 16:21:13.511274 I0115 16:21:13.511274 I0115 16:21:13.511866 I0115 16:21:13.511878 I0115 16:21:13.511275 I0115 16:21:13.511275 I0115 16:25:36.552631 I0115 16:25:36.552774 I0115 16:25:36.552818 I0115 16:25:36.552846 I0115 16:25:36.553839 I0115 16:25:36.554382 I0115 16:25:36.554523 I0115 16:25:36.649437 I0115 16:26:29.602084 I0115 16:26:29.602213 I0115 16:26:29.602224 I0115 16:26:29.602084 I0115 16:26:29.602777 I0115 16:26:29.603232 2025/01/15 16:27:01 I0115 16:30:52.406488 I0115 16:30:52.406526 I0115 16:30:52.406497 I0115 16:30:52.406497 I0115 16:30:52.407230 I0115 16:30:52.407245 I0115 16:34:49.846443 I0115 16:34:49.846620 I0115 16:34:49.846728 I0115 16:34:49.846728 I0115 16:34:49.847504 I0115 16:34:49.847591 I0115 16:34:49.847934 I0115 16:34:49.949283 I0115 16:37:28.318441 I0115 16:37:28.318449 I0115 16:37:28.318959 I0115 16:37:28.319030 I0115 16:37:28.318441 I0115 16:37:28.320329 I0115 16:40:31.302398 I0115 16:40:31.302492 I0115 16:40:31.302509 I0115 16:40:31.302980 I0115 16:40:31.302398 I0115 16:40:31.303575 I0115 16:44:03.139597 I0115 16:44:03.139880 I0115 16:44:03.140812 I0115 16:44:03.140921 I0115 16:44:03.140815 I0115 16:44:03.141436 I0115 16:44:03.142079 I0115 16:44:03.243823 2025/01/15 16:47:02 I0115 16:48:27.033999 I0115 16:48:27.034008 I0115 16:48:27.034015 I0115 16:48:27.034628 I0115 16:48:27.034638 I0115 16:48:27.034015 I0115 16:50:10.197010 I0115 16:50:10.197239 I0115 16:50:10.197010 I0115 16:50:10.197722 I0115 16:50:10.197239 I0115 16:50:10.198307 I0115 16:53:16.433253 I0115 16:53:16.433550 I0115 16:53:16.434244 I0115 16:53:16.435160 I0115 16:53:16.435295 I0115 16:53:16.433548 I0115 16:53:16.436593 I0115 16:53:16.528253 capi-kubeadm-bootstra │ E0115 14:35:01.520580 │ client │ llerGroup="cluster.x-k8s.io" │ reconcileID="add4 │ I0115 14:35:01.758639 │ ntroller="kubeadmconfig" │ -pt9zl" │ ane-pt9zl" │ I0115 14:35:31.520996 │ nd="Cluster" │ 7519220f" │ I0115 14:35:31.760541 │ ntroller="kubeadmconfig" │ -pt9zl" │ ane-pt9zl" │ E0115 14:35:36.569818 capi-kubeadm-control- I0115 14:38:03.063600 │ roup="controlplan │ ame="beta-control-plane" │ ontrol-plane-pt9zl │ E0115 14:38:03.063664 │ client: │ " controllerGroup │ ace="beta" │ I0115 14:38:03.063777 │ control-plane-pt9zl │ ubeadmControlPlan │ cluster.x-k8s.io/ │ I0115 14:38:06.099312 │ nd="Cluster" │ 779027fd" │ E0115 14:38:11.183066 │ client │ llerGroup="cluster.x-k8s.io" │ reconcileID="3776 capi-system/capi-controller-manager E0115 14:39:41.890739 │ client │ llerGroup="cluster.x-k8s.io" │ reconcileID="8615 │ I0115 14:40:11.891501 │ nd="Cluster" │ 32f2912a" │ E0115 14:40:16.945139 │ client │ llerGroup="cluster.x-k8s.io" │ reconcileID="3a9e │ I0115 14:40:46.946072 │ nd="Cluster" │ 19ce9cb6" │ E0115 14:40:51.990007 │ client │ llerGroup="cluster.x-k8s.io" │ reconcileID="52da

capa-controller-manager 1 logger.go:75] "feature gates: AlternativeGCStrategy=false,AutoControllerIdentityCreator=true,BootstrapFormatIgnition=false,EKS=true,EKSAllowAddRoles=false,EKSEnableIAM=false,EKSFargate=false,EventBridgeInstanceState=false,ExternalResourceGC=false,MachinePool=false,ROSA=false,TagUnmanagedNetworkResources=true\n" logger="setup" 1 logger.go:75] "AutoControllerIdentityCreator enabled" logger="setup" 1 webhook.go:177] "skip registering a mutating webhook, object does not implement admission.Defaulter or WithDefaulter wasn't called" logger="controller-runtime.builder" GVK="infrastructure.cluster.x-k8s.io/v1beta2, Kind=AWSMachineTemplate" 1 webhook.go:193] "Registering a validating webhook" logger="controller-runtime.builder" GVK="infrastructure.cluster.x-k8s.io/v1beta2, Kind=AWSMachineTemplate" path="/validate-infrastructure-cluster-x-k8s-io-v1beta2-awsmachinetemplate" 1 server.go:183] "Registering webhook" logger="controller-runtime.webhook" path="/validate-infrastructure-cluster-x-k8s-io-v1beta2-awsmachinetemplate" 1 server.go:183] "Registering webhook" logger="controller-runtime.webhook" path="/convert" 1 webhook.go:224] "Conversion webhook enabled" logger="controller-runtime.builder" GVK="infrastructure.cluster.x-k8s.io/v1beta2, Kind=AWSMachineTemplate" 1 webhook.go:162] "Registering a mutating webhook" logger="controller-runtime.builder" GVK="infrastructure.cluster.x-k8s.io/v1beta2, Kind=AWSCluster" path="/mutate-infrastructure-cluster-x-k8s-io-v1beta2-awscluster" 1 server.go:183] "Registering webhook" logger="controller-runtime.webhook" path="/mutate-infrastructure-cluster-x-k8s-io-v1beta2-awscluster" 1 webhook.go:193] "Registering a validating webhook" logger="controller-runtime.builder" GVK="infrastructure.cluster.x-k8s.io/v1beta2, Kind=AWSCluster" path="/validate-infrastructure-cluster-x-k8s-io-v1beta2-awscluster" 1 server.go:183] "Registering webhook" logger="controller-runtime.webhook" path="/validate-infrastructure-cluster-x-k8s-io-v1beta2-awscluster" 1 webhook.go:224] "Conversion webhook enabled" logger="controller-runtime.builder" GVK="infrastructure.cluster.x-k8s.io/v1beta2, Kind=AWSCluster" 1 webhook.go:162] "Registering a mutating webhook" logger="controller-runtime.builder" GVK="infrastructure.cluster.x-k8s.io/v1beta2, Kind=AWSClusterTemplate" path="/mutate-infrastructure-cluster-x-k8s-io-v1beta2-awsclustertemplate" 1 server.go:183] "Registering webhook" logger="controller-runtime.webhook" path="/mutate-infrastructure-cluster-x-k8s-io-v1beta2-awsclustertemplate" 1 webhook.go:193] "Registering a validating webhook" logger="controller-runtime.builder" GVK="infrastructure.cluster.x-k8s.io/v1beta2, Kind=AWSClusterTemplate" path="/validate-infrastructure-cluster-x-k8s-io-v1beta2-awsclustertemplate" 1 server.go:183] "Registering webhook" logger="controller-runtime.webhook" path="/validate-infrastructure-cluster-x-k8s-io-v1beta2-awsclustertemplate" 1 webhook.go:224] "Conversion webhook enabled" logger="controller-runtime.builder" GVK="infrastructure.cluster.x-k8s.io/v1beta2, Kind=AWSClusterTemplate" 1 webhook.go:162] "Registering a mutating webhook" logger="controller-runtime.builder" GVK="infrastructure.cluster.x-k8s.io/v1beta2, Kind=AWSClusterControllerIdentity" path="/mutate-infrastructure-cluster-x-k8s-io-v1beta2-awsclustercontrolleridentity" 1 server.go:183] "Registering webhook" logger="controller-runtime.webhook" path="/mutate-infrastructure-cluster-x-k8s-io-v1beta2-awsclustercontrolleridentity" 1 webhook.go:193] "Registering a validating webhook" logger="controller-runtime.builder" GVK="infrastructure.cluster.x-k8s.io/v1beta2, Kind=AWSClusterControllerIdentity" path="/validate-infrastructure-cluster-x-k8s-io-v1beta2-awsclustercontrolleridentity" 1 server.go:183] "Registering webhook" logger="controller-runtime.webhook" path="/validate-infrastructure-cluster-x-k8s-io-v1beta2-awsclustercontrolleridentity" 1 webhook.go:224] "Conversion webhook enabled" logger="controller-runtime.builder" GVK="infrastructure.cluster.x-k8s.io/v1beta2, Kind=AWSClusterControllerIdentity" 1 webhook.go:162] "Registering a mutating webhook" logger="controller-runtime.builder" GVK="infrastructure.cluster.x-k8s.io/v1beta2, Kind=AWSClusterRoleIdentity" path="/mutate-infrastructure-cluster-x-k8s-io-v1beta2-awsclusterroleidentity" 1 server.go:183] "Registering webhook" logger="controller-runtime.webhook" path="/mutate-infrastructure-cluster-x-k8s-io-v1beta2-awsclusterroleidentity" 1 webhook.go:193] "Registering a validating webhook" logger="controller-runtime.builder" GVK="infrastructure.cluster.x-k8s.io/v1beta2, Kind=AWSClusterRoleIdentity" path="/validate-infrastructure-cluster-x-k8s-io-v1beta2-awsclusterroleidentity" 1 server.go:183] "Registering webhook" logger="controller-runtime.webhook" path="/validate-infrastructure-cluster-x-k8s-io-v1beta2-awsclusterroleidentity" 1 webhook.go:224] "Conversion webhook enabled" logger="controller-runtime.builder" GVK="infrastructure.cluster.x-k8s.io/v1beta2, Kind=AWSClusterRoleIdentity" 1 webhook.go:162] "Registering a mutating webhook" logger="controller-runtime.builder" GVK="infrastructure.cluster.x-k8s.io/v1beta2, Kind=AWSClusterStaticIdentity" path="/mutate-infrastructure-cluster-x-k8s-io-v1beta2-awsclusterstaticidentity" 1 server.go:183] "Registering webhook" logger="controller-runtime.webhook" path="/mutate-infrastructure-cluster-x-k8s-io-v1beta2-awsclusterstaticidentity" 1 webhook.go:193] "Registering a validating webhook" logger="controller-runtime.builder" GVK="infrastructure.cluster.x-k8s.io/v1beta2, Kind=AWSClusterStaticIdentity" path="/validate-infrastructure-cluster-x-k8s-io-v1beta2-awsclusterstaticidentity" 1 server.go:183] "Registering webhook" logger="controller-runtime.webhook" path="/validate-infrastructure-cluster-x-k8s-io-v1beta2-awsclusterstaticidentity" 1 webhook.go:224] "Conversion webhook enabled" logger="controller-runtime.builder" GVK="infrastructure.cluster.x-k8s.io/v1beta2, Kind=AWSClusterStaticIdentity" 1 webhook.go:162] "Registering a mutating webhook" logger="controller-runtime.builder" GVK="infrastructure.cluster.x-k8s.io/v1beta2, Kind=AWSMachine" path="/mutate-infrastructure-cluster-x-k8s-io-v1beta2-awsmachine" 1 server.go:183] "Registering webhook" logger="controller-runtime.webhook" path="/mutate-infrastructure-cluster-x-k8s-io-v1beta2-awsmachine" 1 webhook.go:193] "Registering a validating webhook" logger="controller-runtime.builder" GVK="infrastructure.cluster.x-k8s.io/v1beta2, Kind=AWSMachine" path="/validate-infrastructure-cluster-x-k8s-io-v1beta2-awsmachine" 1 server.go:183] "Registering webhook" logger="controller-runtime.webhook" path="/validate-infrastructure-cluster-x-k8s-io-v1beta2-awsmachine" 1 webhook.go:224] "Conversion webhook enabled" logger="controller-runtime.builder" GVK="infrastructure.cluster.x-k8s.io/v1beta2, Kind=AWSMachine" 1 logger.go:75] "enabling EKS controllers and webhooks" logger="setup" 1 webhook.go:162] "Registering a mutating webhook" logger="controller-runtime.builder" GVK="controlplane.cluster.x-k8s.io/v1beta2, Kind=AWSManagedControlPlane" path="/mutate-controlplane-cluster-x-k8s-io-v1beta2-awsmanagedcontrolplane" 1 server.go:183] "Registering webhook" logger="controller-runtime.webhook" path="/mutate-controlplane-cluster-x-k8s-io-v1beta2-awsmanagedcontrolplane" 1 webhook.go:193] "Registering a validating webhook" logger="controller-runtime.builder" GVK="controlplane.cluster.x-k8s.io/v1beta2, Kind=AWSManagedControlPlane" path="/validate-controlplane-cluster-x-k8s-io-v1beta2-awsmanagedcontrolplane" 1 server.go:183] "Registering webhook" logger="controller-runtime.webhook" path="/validate-controlplane-cluster-x-k8s-io-v1beta2-awsmanagedcontrolplane" 1 webhook.go:224] "Conversion webhook enabled" logger="controller-runtime.builder" GVK="controlplane.cluster.x-k8s.io/v1beta2, Kind=AWSManagedControlPlane" 1 logger.go:75] "starting manager" logger="setup" version="v2.7.1" 1 server.go:208] "Starting metrics server" logger="controller-runtime.metrics" 1 server.go:83] "starting server" name="health probe" addr="[::]:9440" 1 server.go:191] "Starting webhook server" logger="controller-runtime.webhook" 1 certwatcher.go:161] "Updated current TLS certificate" logger="controller-runtime.certwatcher" 1 server.go:242] "Serving webhook server" logger="controller-runtime.webhook" host="" port=9443 1 certwatcher.go:115] "Starting certificate watcher" logger="controller-runtime.certwatcher" 1 leaderelection.go:250] attempting to acquire leader lease capa-system/controller-leader-elect-capa... 1 server.go:247] "Serving metrics server" logger="controller-runtime.metrics" bindAddress=":8443" secure=true http: TLS handshake error from 10.XXX.X.1.105:48570: EOF 1 leaderelection.go:260] successfully acquired lease capa-system/controller-leader-elect-capa 1 controller.go:173] "Starting EventSource" controller="awsmachine" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSMachine" source="kind source: *v1beta2.AWSMachine" 1 controller.go:173] "Starting EventSource" controller="awsmachine" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSMachine" source="kind source: *v1beta1.Machine" 1 controller.go:173] "Starting EventSource" controller="awsmachine" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSMachine" source="kind source: *v1beta2.AWSCluster" 1 controller.go:173] "Starting EventSource" controller="awsmachine" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSMachine" source="kind source: *v1beta1.Cluster" 1 controller.go:181] "Starting Controller" controller="awsmachine" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSMachine" 1 controller.go:173] "Starting EventSource" controller="awscluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSCluster" source="kind source: *v1beta2.AWSCluster" 1 controller.go:173] "Starting EventSource" controller="awscluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSCluster" source="kind source: *v1beta2.AWSManagedControlPlane" 1 controller.go:181] "Starting Controller" controller="awscluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSCluster" 1 controller.go:173] "Starting EventSource" controller="awscluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSCluster" source="kind source: *v1beta2.AWSCluster" 1 controller.go:173] "Starting EventSource" controller="awscluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSCluster" source="kind source: *v1beta1.Cluster" 1 controller.go:181] "Starting Controller" controller="awscluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSCluster" 1 controller.go:173] "Starting EventSource" controller="eksconfig" controllerGroup="bootstrap.cluster.x-k8s.io" controllerKind="EKSConfig" source="kind source: *v1beta2.EKSConfig" 1 controller.go:173] "Starting EventSource" controller="eksconfig" controllerGroup="bootstrap.cluster.x-k8s.io" controllerKind="EKSConfig" source="kind source: *v1beta1.Machine" 1 controller.go:173] "Starting EventSource" controller="eksconfig" controllerGroup="bootstrap.cluster.x-k8s.io" controllerKind="EKSConfig" source="kind source: *v1beta1.Cluster" 1 controller.go:181] "Starting Controller" controller="eksconfig" controllerGroup="bootstrap.cluster.x-k8s.io" controllerKind="EKSConfig" 1 controller.go:173] "Starting EventSource" controller="awsmanagedcontrolplane" controllerGroup="controlplane.cluster.x-k8s.io" controllerKind="AWSManagedControlPlane" source="kind source: *v1beta2.AWSManagedControlPlane" 1 controller.go:173] "Starting EventSource" controller="awsmanagedcontrolplane" controllerGroup="controlplane.cluster.x-k8s.io" controllerKind="AWSManagedControlPlane" source="kind source: *v1beta1.Cluster" 1 controller.go:173] "Starting EventSource" controller="awsmanagedcontrolplane" controllerGroup="controlplane.cluster.x-k8s.io" controllerKind="AWSManagedControlPlane" source="kind source: *v1beta2.AWSManagedCluster" 1 controller.go:181] "Starting Controller" controller="awsmanagedcontrolplane" controllerGroup="controlplane.cluster.x-k8s.io" controllerKind="AWSManagedControlPlane" 1 controller.go:173] "Starting EventSource" controller="awsmanagedcluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSManagedCluster" source="kind source: *v1beta2.AWSManagedCluster" 1 controller.go:173] "Starting EventSource" controller="awsmanagedcluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSManagedCluster" source="kind source: *v1beta1.Cluster" 1 controller.go:173] "Starting EventSource" controller="awsmanagedcluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSManagedCluster" source="kind source: *v1beta2.AWSManagedControlPlane" 1 controller.go:181] "Starting Controller" controller="awsmanagedcluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSManagedCluster" 1 controller.go:215] "Starting workers" controller="awsmachine" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSMachine" worker count=10 1 controller.go:215] "Starting workers" controller="awsmanagedcontrolplane" controllerGroup="controlplane.cluster.x-k8s.io" controllerKind="AWSManagedControlPlane" worker count=5 1 controller.go:215] "Starting workers" controller="awsmanagedcluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSManagedCluster" worker count=5 1 controller.go:215] "Starting workers" controller="awscluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSCluster" worker count=5 1 controller.go:215] "Starting workers" controller="eksconfig" controllerGroup="bootstrap.cluster.x-k8s.io" controllerKind="EKSConfig" worker count=5 1 controller.go:215] "Starting workers" controller="awscluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSCluster" worker count=5 1 awscluster_controller.go:301] "Reconciling AWSCluster" controller="awscluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSCluster" AWSCluster="beta/beta-aws-workload-cluster" namespace="beta" name="beta-aws-workload-cluster" reconcileID="a1d24320-6122-4a20-bfe5-f2583f9190fe" cluster="beta/beta-workload-cluster" 1 warning_handler.go:65] "metadata.finalizers: \"awscluster.infrastructure.cluster.x-k8s.io\": prefer a domain-qualified finalizer name to avoid accidental conflicts with other finalizer writers" logger="KubeAPIWarningLogger" 1 subnets.go:50] "Reconciling subnets" controller="awscluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSCluster" AWSCluster="beta/beta-aws-workload-cluster" namespace="beta" name="beta-aws-workload-cluster" reconcileID="a1d24320-6122-4a20-bfe5-f2583f9190fe" cluster="beta/beta-workload-cluster" 1 loadbalancer.go:1190] "Created classic load balancer" controller="awscluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSCluster" AWSCluster="beta/beta-aws-workload-cluster" namespace="beta" name="beta-aws-workload-cluster" reconcileID="a1d24320-6122-4a20-bfe5-f2583f9190fe" cluster="beta/beta-workload-cluster" dns-name="internal-beta-workload-cluster-apiserver-1675643417.us-west-1.elb.amazonaws.com" 1 awscluster_controller.go:301] "Reconciling AWSCluster" controller="awscluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSCluster" AWSCluster="beta/beta-aws-workload-cluster" namespace="beta" name="beta-aws-workload-cluster" reconcileID="71985743-9ac4-4637-a479-f8971767d93f" cluster="beta/beta-workload-cluster" 1 subnets.go:50] "Reconciling subnets" controller="awscluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSCluster" AWSCluster="beta/beta-aws-workload-cluster" namespace="beta" name="beta-aws-workload-cluster" reconcileID="71985743-9ac4-4637-a479-f8971767d93f" cluster="beta/beta-workload-cluster" 1 awscluster_controller.go:301] "Reconciling AWSCluster" controller="awscluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSCluster" AWSCluster="beta/beta-aws-workload-cluster" namespace="beta" name="beta-aws-workload-cluster" reconcileID="c8b316ed-02ed-4d38-87a6-4f27f4b56868" cluster="beta/beta-workload-cluster" 1 subnets.go:50] "Reconciling subnets" controller="awscluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSCluster" AWSCluster="beta/beta-aws-workload-cluster" namespace="beta" name="beta-aws-workload-cluster" reconcileID="c8b316ed-02ed-4d38-87a6-4f27f4b56868" cluster="beta/beta-workload-cluster" 1 awscluster_controller.go:301] "Reconciling AWSCluster" controller="awscluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSCluster" AWSCluster="beta/beta-aws-workload-cluster" namespace="beta" name="beta-aws-workload-cluster" reconcileID="0772be47-4374-40b1-9f47-895a270f9177" cluster="beta/beta-workload-cluster" 1 subnets.go:50] "Reconciling subnets" controller="awscluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSCluster" AWSCluster="beta/beta-aws-workload-cluster" namespace="beta" name="beta-aws-workload-cluster" reconcileID="0772be47-4374-40b1-9f47-895a270f9177" cluster="beta/beta-workload-cluster" 1 awsmachine_controller.go:173] "Machine Controller has not yet set OwnerRef" controller="awsmachine" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSMachine" AWSMachine="beta/beta-control-plane-pt9zl" namespace="beta" name="beta-control-plane-pt9zl" reconcileID="9274c17d-68e4-4f13-8868-e67a6ba68612" 1 awsmachine_controller.go:173] "Machine Controller has not yet set OwnerRef" controller="awsmachine" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSMachine" AWSMachine="beta/beta-control-plane-pt9zl" namespace="beta" name="beta-control-plane-pt9zl" reconcileID="76107800-d9a7-403f-9827-7c2ca46651c1" 1 awsmachine_controller.go:173] "Machine Controller has not yet set OwnerRef" controller="awsmachine" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSMachine" AWSMachine="beta/beta-control-plane-pt9zl" namespace="beta" name="beta-control-plane-pt9zl" reconcileID="e195d3f1-adf5-4d94-9eb7-c1a6244dc10a" 1 awsmachine_controller.go:173] "Machine Controller has not yet set OwnerRef" controller="awsmachine" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSMachine" AWSMachine="beta/beta-control-plane-pt9zl" namespace="beta" name="beta-control-plane-pt9zl" reconcileID="23a5ed53-7c4d-4a3f-a933-1f128b09f971" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:173] "Machine Controller has not yet set OwnerRef" controller="awsmachine" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSMachine" AWSMachine="beta/beta-dbbases-deployment-w8llg-s6nzn" namespace="beta" name="beta-dbbases-deployment-w8llg-s6nzn" reconcileID="a849a6f5-6571-4986-8f23-24e20fbd6e7a" 1 awsmachine_controller.go:173] "Machine Controller has not yet set OwnerRef" controller="awsmachine" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSMachine" AWSMachine="beta/beta-dbbases-deployment-w8llg-s6nzn" namespace="beta" name="beta-dbbases-deployment-w8llg-s6nzn" reconcileID="06029063-60b5-4ae7-96de-c78cce44ab29" 1 awsmachine_controller.go:173] "Machine Controller has not yet set OwnerRef" controller="awsmachine" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSMachine" AWSMachine="beta/beta-dbbases-deployment-w8llg-s6nzn" namespace="beta" name="beta-dbbases-deployment-w8llg-s6nzn" reconcileID="17405193-5684-433e-8aca-0bfd86491824" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:173] "Machine Controller has not yet set OwnerRef" controller="awsmachine" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSMachine" AWSMachine="beta/beta-worker-deployment-g9r6v-fg7ql" namespace="beta" name="beta-worker-deployment-g9r6v-fg7ql" reconcileID="902c26db-8a55-47c2-b50b-285ea6c502b2" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:173] "Machine Controller has not yet set OwnerRef" controller="awsmachine" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSMachine" AWSMachine="beta/beta-worker-deployment-g9r6v-fg7ql" namespace="beta" name="beta-worker-deployment-g9r6v-fg7ql" reconcileID="254ed2c7-37f1-4a4e-bca5-9d5bfd13605b" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:173] "Machine Controller has not yet set OwnerRef" controller="awsmachine" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSMachine" AWSMachine="beta/beta-worker-deployment-g9r6v-fg7ql" namespace="beta" name="beta-worker-deployment-g9r6v-fg7ql" reconcileID="46bde8d0-06d7-4910-8f10-12244310d79a" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" http: TLS handshake error from 10.XXX.X.1.105:36938: EOF 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:173] "Machine Controller has not yet set OwnerRef" controller="awsmachine" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSMachine" AWSMachine="beta/beta-dbservices-deployment-rxvdh-njm9w" namespace="beta" name="beta-dbservices-deployment-rxvdh-njm9w" reconcileID="09844918-833f-4aef-99a4-ad6584763676" 1 awsmachine_controller.go:173] "Machine Controller has not yet set OwnerRef" controller="awsmachine" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSMachine" AWSMachine="beta/beta-labservices-deployment-zdbfb-phhsm" namespace="beta" name="beta-labservices-deployment-zdbfb-phhsm" reconcileID="12e7f7d1-4400-40cb-9740-aae3084673c6" 1 awsmachine_controller.go:173] "Machine Controller has not yet set OwnerRef" controller="awsmachine" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSMachine" AWSMachine="beta/beta-performance-gpu-deployment-qwd6x-h2b97" namespace="beta" name="beta-performance-gpu-deployment-qwd6x-h2b97" reconcileID="4ceeadae-f4a5-421e-801a-2ad02d97f5ce" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:173] "Machine Controller has not yet set OwnerRef" controller="awsmachine" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSMachine" AWSMachine="beta/beta-dbservices-deployment-rxvdh-njm9w" namespace="beta" name="beta-dbservices-deployment-rxvdh-njm9w" reconcileID="310ef7d1-bc18-413b-b076-9e734d725c91" 1 awsmachine_controller.go:173] "Machine Controller has not yet set OwnerRef" controller="awsmachine" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSMachine" AWSMachine="beta/beta-labservices-deployment-zdbfb-phhsm" namespace="beta" name="beta-labservices-deployment-zdbfb-phhsm" reconcileID="5c872049-9c50-4ec9-8fa6-0d218f4ad85e" 1 awsmachine_controller.go:173] "Machine Controller has not yet set OwnerRef" controller="awsmachine" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSMachine" AWSMachine="beta/beta-performance-gpu-deployment-qwd6x-h2b97" namespace="beta" name="beta-performance-gpu-deployment-qwd6x-h2b97" reconcileID="03ffdc01-1d1d-4e87-a9dd-16e45d8a3522" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:173] "Machine Controller has not yet set OwnerRef" controller="awsmachine" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSMachine" AWSMachine="beta/beta-dbservices-deployment-rxvdh-njm9w" namespace="beta" name="beta-dbservices-deployment-rxvdh-njm9w" reconcileID="af8f32c8-234c-4a8d-af23-a7fd6ae9264b" 1 awsmachine_controller.go:173] "Machine Controller has not yet set OwnerRef" controller="awsmachine" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSMachine" AWSMachine="beta/beta-performance-gpu-deployment-qwd6x-h2b97" namespace="beta" name="beta-performance-gpu-deployment-qwd6x-h2b97" reconcileID="b2d21c14-73c8-4b1c-ba76-ecab2fcfc44f" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:173] "Machine Controller has not yet set OwnerRef" controller="awsmachine" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSMachine" AWSMachine="beta/beta-labservices-deployment-zdbfb-phhsm" namespace="beta" name="beta-labservices-deployment-zdbfb-phhsm" reconcileID="6bd88b82-eb09-4cb1-97c0-d2ea3923cd42" 1 awsmachine_controller.go:173] "Machine Controller has not yet set OwnerRef" controller="awsmachine" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSMachine" AWSMachine="beta/beta-dbservices-deployment-rxvdh-njm9w" namespace="beta" name="beta-dbservices-deployment-rxvdh-njm9w" reconcileID="0ee1ec88-2e41-43ae-b763-4f754cd4542c" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awscluster_controller.go:301] "Reconciling AWSCluster" controller="awscluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSCluster" AWSCluster="beta/beta-aws-workload-cluster" namespace="beta" name="beta-aws-workload-cluster" reconcileID="6b2039cf-0b31-4b59-bab1-c4f26b05b484" cluster="beta/beta-workload-cluster" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 subnets.go:50] "Reconciling subnets" controller="awscluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSCluster" AWSCluster="beta/beta-aws-workload-cluster" namespace="beta" name="beta-aws-workload-cluster" reconcileID="6b2039cf-0b31-4b59-bab1-c4f26b05b484" cluster="beta/beta-workload-cluster" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awscluster_controller.go:301] "Reconciling AWSCluster" controller="awscluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSCluster" AWSCluster="beta/beta-aws-workload-cluster" namespace="beta" name="beta-aws-workload-cluster" reconcileID="9acd7924-36f9-4228-bc35-d6983b81c1ee" cluster="beta/beta-workload-cluster" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 subnets.go:50] "Reconciling subnets" controller="awscluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSCluster" AWSCluster="beta/beta-aws-workload-cluster" namespace="beta" name="beta-aws-workload-cluster" reconcileID="9acd7924-36f9-4228-bc35-d6983b81c1ee" cluster="beta/beta-workload-cluster" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awscluster_controller.go:301] "Reconciling AWSCluster" controller="awscluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSCluster" AWSCluster="beta/beta-aws-workload-cluster" namespace="beta" name="beta-aws-workload-cluster" reconcileID="962fbd79-c5d0-43bf-a521-455b28947e20" cluster="beta/beta-workload-cluster" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 subnets.go:50] "Reconciling subnets" controller="awscluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSCluster" AWSCluster="beta/beta-aws-workload-cluster" namespace="beta" name="beta-aws-workload-cluster" reconcileID="962fbd79-c5d0-43bf-a521-455b28947e20" cluster="beta/beta-workload-cluster" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" http: TLS handshake error from 10.XXX.X.1.105:50810: EOF 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awscluster_controller.go:301] "Reconciling AWSCluster" controller="awscluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSCluster" AWSCluster="beta/beta-aws-workload-cluster" namespace="beta" name="beta-aws-workload-cluster" reconcileID="9b153d87-d1c9-4724-8b9c-6956b91da13a" cluster="beta/beta-workload-cluster" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 subnets.go:50] "Reconciling subnets" controller="awscluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSCluster" AWSCluster="beta/beta-aws-workload-cluster" namespace="beta" name="beta-aws-workload-cluster" reconcileID="9b153d87-d1c9-4724-8b9c-6956b91da13a" cluster="beta/beta-workload-cluster" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awscluster_controller.go:301] "Reconciling AWSCluster" controller="awscluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSCluster" AWSCluster="beta/beta-aws-workload-cluster" namespace="beta" name="beta-aws-workload-cluster" reconcileID="138ececb-b250-49f1-b314-7347970d1fd1" cluster="beta/beta-workload-cluster" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 subnets.go:50] "Reconciling subnets" controller="awscluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSCluster" AWSCluster="beta/beta-aws-workload-cluster" namespace="beta" name="beta-aws-workload-cluster" reconcileID="138ececb-b250-49f1-b314-7347970d1fd1" cluster="beta/beta-workload-cluster" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awscluster_controller.go:301] "Reconciling AWSCluster" controller="awscluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSCluster" AWSCluster="beta/beta-aws-workload-cluster" namespace="beta" name="beta-aws-workload-cluster" reconcileID="9b8e5604-0a9f-4d9a-a3b2-ab8897cd8f0c" cluster="beta/beta-workload-cluster" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 subnets.go:50] "Reconciling subnets" controller="awscluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSCluster" AWSCluster="beta/beta-aws-workload-cluster" namespace="beta" name="beta-aws-workload-cluster" reconcileID="9b8e5604-0a9f-4d9a-a3b2-ab8897cd8f0c" cluster="beta/beta-workload-cluster" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awscluster_controller.go:301] "Reconciling AWSCluster" controller="awscluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSCluster" AWSCluster="beta/beta-aws-workload-cluster" namespace="beta" name="beta-aws-workload-cluster" reconcileID="5bcfd39a-2aa9-4f04-812a-6d6c9056353c" cluster="beta/beta-workload-cluster" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 subnets.go:50] "Reconciling subnets" controller="awscluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSCluster" AWSCluster="beta/beta-aws-workload-cluster" namespace="beta" name="beta-aws-workload-cluster" reconcileID="5bcfd39a-2aa9-4f04-812a-6d6c9056353c" cluster="beta/beta-workload-cluster" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" http: TLS handshake error from 10.XXX.X.1.105:38944: EOF http: TLS handshake error from 10.XXX.X.1.105:38942: EOF 1 awscluster_controller.go:301] "Reconciling AWSCluster" controller="awscluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSCluster" AWSCluster="beta/beta-aws-workload-cluster" namespace="beta" name="beta-aws-workload-cluster" reconcileID="e422a102-4641-4d6b-be73-0e69ad1c7989" cluster="beta/beta-workload-cluster" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 subnets.go:50] "Reconciling subnets" controller="awscluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSCluster" AWSCluster="beta/beta-aws-workload-cluster" namespace="beta" name="beta-aws-workload-cluster" reconcileID="e422a102-4641-4d6b-be73-0e69ad1c7989" cluster="beta/beta-workload-cluster" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awscluster_controller.go:301] "Reconciling AWSCluster" controller="awscluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSCluster" AWSCluster="beta/beta-aws-workload-cluster" namespace="beta" name="beta-aws-workload-cluster" reconcileID="cc00a954-0a76-400e-ae11-d20018b0fe83" cluster="beta/beta-workload-cluster" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 subnets.go:50] "Reconciling subnets" controller="awscluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSCluster" AWSCluster="beta/beta-aws-workload-cluster" namespace="beta" name="beta-aws-workload-cluster" reconcileID="cc00a954-0a76-400e-ae11-d20018b0fe83" cluster="beta/beta-workload-cluster" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" http: TLS handshake error from 10.XXX.X.5:41740: EOF 1 awscluster_controller.go:301] "Reconciling AWSCluster" controller="awscluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSCluster" AWSCluster="beta/beta-aws-workload-cluster" namespace="beta" name="beta-aws-workload-cluster" reconcileID="7f614361-c241-4b2d-a7c1-c2fa88fa35f2" cluster="beta/beta-workload-cluster" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 subnets.go:50] "Reconciling subnets" controller="awscluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSCluster" AWSCluster="beta/beta-aws-workload-cluster" namespace="beta" name="beta-aws-workload-cluster" reconcileID="7f614361-c241-4b2d-a7c1-c2fa88fa35f2" cluster="beta/beta-workload-cluster" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awscluster_controller.go:301] "Reconciling AWSCluster" controller="awscluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSCluster" AWSCluster="beta/beta-aws-workload-cluster" namespace="beta" name="beta-aws-workload-cluster" reconcileID="71117995-5f1c-40ab-b8a1-5e4f01ef1b64" cluster="beta/beta-workload-cluster" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 subnets.go:50] "Reconciling subnets" controller="awscluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSCluster" AWSCluster="beta/beta-aws-workload-cluster" namespace="beta" name="beta-aws-workload-cluster" reconcileID="71117995-5f1c-40ab-b8a1-5e4f01ef1b64" cluster="beta/beta-workload-cluster" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awscluster_controller.go:301] "Reconciling AWSCluster" controller="awscluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSCluster" AWSCluster="beta/beta-aws-workload-cluster" namespace="beta" name="beta-aws-workload-cluster" reconcileID="c651e7d7-7613-4a68-a255-063450a75f51" cluster="beta/beta-workload-cluster" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 subnets.go:50] "Reconciling subnets" controller="awscluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSCluster" AWSCluster="beta/beta-aws-workload-cluster" namespace="beta" name="beta-aws-workload-cluster" reconcileID="c651e7d7-7613-4a68-a255-063450a75f51" cluster="beta/beta-workload-cluster" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" http: TLS handshake error from 10.XXX.X.1.105:44112: EOF 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awscluster_controller.go:301] "Reconciling AWSCluster" controller="awscluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSCluster" AWSCluster="beta/beta-aws-workload-cluster" namespace="beta" name="beta-aws-workload-cluster" reconcileID="67c242c5-0e49-4359-872f-a9e8b640a943" cluster="beta/beta-workload-cluster" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 subnets.go:50] "Reconciling subnets" controller="awscluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSCluster" AWSCluster="beta/beta-aws-workload-cluster" namespace="beta" name="beta-aws-workload-cluster" reconcileID="67c242c5-0e49-4359-872f-a9e8b640a943" cluster="beta/beta-workload-cluster" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awscluster_controller.go:301] "Reconciling AWSCluster" controller="awscluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSCluster" AWSCluster="beta/beta-aws-workload-cluster" namespace="beta" name="beta-aws-workload-cluster" reconcileID="d34b67e1-f91f-4d5a-bdd1-ac5d9b653434" cluster="beta/beta-workload-cluster" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 subnets.go:50] "Reconciling subnets" controller="awscluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSCluster" AWSCluster="beta/beta-aws-workload-cluster" namespace="beta" name="beta-aws-workload-cluster" reconcileID="d34b67e1-f91f-4d5a-bdd1-ac5d9b653434" cluster="beta/beta-workload-cluster" http: TLS handshake error from 10.XXX.X.1.105:56712: EOF 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awscluster_controller.go:301] "Reconciling AWSCluster" controller="awscluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSCluster" AWSCluster="beta/beta-aws-workload-cluster" namespace="beta" name="beta-aws-workload-cluster" reconcileID="96c01881-42bb-4918-ab1c-6c6d0c655c45" cluster="beta/beta-workload-cluster" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 awsmachine_controller.go:486] "Bootstrap data secret reference is not yet available" 1 subnets.go:50] "Reconciling subnets" controller="awscluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSCluster" AWSCluster="beta/beta-aws-workload-cluster" namespace="beta" name="beta-aws-workload-cluster" reconcileID="96c01881-42bb-4918-ab1c-6c6d0c655c45" cluster="beta/beta-workload-cluster" p-system/capi-kubeadm-bootstrap-controller-manager │ 1 cluster_accessor.go:257] "Connect failed" err="error creating HTTP client and mapper: cluster is not reachable: │ rate limiter Wait returned an error: context deadline exceeded - error from a previous attempt: EOF" controller="clustercache" contro │ controllerKind="Cluster" Cluster="beta/beta-workload-cluster" namespace="beta" name="beta-workload-cluster"  │ 5828-17aa-42cf-ac54-4a9ae693f341"                                                                                            │ 1 kubeadmconfig_controller.go:466] "Control plane is not ready, requeuing joining control planes until ready." co │ controllerGroup="bootstrap.cluster.x-k8s.io" controllerKind="KubeadmConfig" KubeadmConfig="beta/beta-control-plane │ namespace="beta" name="beta-control-plane-pt9zl" reconcileID="fe94313a-375a-4963-8d9f-982243b8af7a" Machine="beta/beta-control-pl │ resourceVersion="2157785" KubeadmControlPlane="beta/beta-control-plane" Cluster="beta/beta-workload-cluster"                   │ 1 cluster_accessor.go:247] "Connecting" controller="clustercache" controllerGroup="cluster.x-k8s.io" controllerKi │ Cluster="beta/beta-workload-cluster" namespace="beta" name="beta-workload-cluster" reconcileID="412e6b4c-ff80-430d-a1bf-db42 │ │ 1 kubeadmconfig_controller.go:466] "Control plane is not ready, requeuing joining control planes until ready." co │ controllerGroup="bootstrap.cluster.x-k8s.io" controllerKind="KubeadmConfig" KubeadmConfig="beta/beta-control-plane │ namespace="beta" name="beta-control-plane-pt9zl" reconcileID="cc5ea3ca-ae93-4fbe-9b3a-90524b1f18e8" Machine="beta/beta-control-pl │ resourceVersion="2157785" KubeadmControlPlane="beta/beta-control-plane" Cluster="beta/beta-workload-cluster"                   │ 1 cluster_accessor.go:257] "Connect failed" err="error creating HTTP client and mapper: cluster is not reachable: plane-system/capi-kubeadm-control-plane-controller-manager 1 scale.go:229] "Waiting for control plane to pass preflight checks" controller="kubeadmcontrolplane" controllerG │ e.cluster.x-k8s.io" controllerKind="KubeadmControlPlane" KubeadmControlPlane="beta/beta-control-plane" namespace="beta" n │ reconcileID="ea458994-5928-4321-9eea-b7cae9a0743c" Cluster="beta/beta-workload-cluster" failures="Machine beta-c │ does not have a corresponding Node yet (Machine.status.nodeRef not set)"                                                   │ 1 controller.go:220] "Could not connect to workload cluster to fetch status" err="failed to create remote cluster │ beta/beta-workload-cluster: error getting REST config: connection to the workload cluster is down" controller="kubeadmcontrolplane │ ="controlplane.cluster.x-k8s.io" controllerKind="KubeadmControlPlane" KubeadmControlPlane="beta/beta-control-plane" namesp │ name="beta-control-plane" reconcileID="ea458994-5928-4321-9eea-b7cae9a0743c" Cluster="beta/beta-workload-cluster"              │ 1 recorder.go:104] "Waiting for control plane to pass preflight checks to continue reconciliation: Machine beta- │ does not have a corresponding Node yet (Machine.status.nodeRef not set)" logger="events" type="Warning" object={"kind":"K │ e","namespace":"beta","name":"beta-control-plane","uid":"2268be95-db66-4ed4-ad0e-56ee7872a1a8","apiVersion":"controlplane. │ v1beta1","resourceVersion":"2157793"} reason="ControlPlaneUnhealthy"                                                         │ 1 cluster_accessor.go:247] "Connecting" controller="clustercache" controllerGroup="cluster.x-k8s.io" controllerKi │ Cluster="beta/beta-workload-cluster" namespace="beta" name="beta-workload-cluster" reconcileID="37767551-621f-46b5-8a88-bf65 │ │ 1 cluster_accessor.go:257] "Connect failed" err="error creating HTTP client and mapper: cluster is not reachable: │ rate limiter Wait returned an error: context deadline exceeded - error from a previous attempt: EOF" controller="clustercache" contro │ controllerKind="Cluster" Cluster="beta/beta-workload-cluster" namespace="beta" name="beta-workload-cluster"  │ 7551-621f-46b5-8a88-bf65779027fd" 1 cluster_accessor.go:257] "Connect failed" err="error creating HTTP client and mapper: cluster is not reachable: │ rate limiter Wait returned an error: context deadline exceeded - error from a previous attempt: EOF" controller="clustercache" contro │ controllerKind="Cluster" Cluster="beta/beta-workload-cluster" namespace="beta" name="beta-workload-cluster"  │ ffa3-edbf-4672-86ba-486ad04fbdc8"                                                                                            │ 1 cluster_accessor.go:247] "Connecting" controller="clustercache" controllerGroup="cluster.x-k8s.io" controllerKi │ Cluster="beta/beta-workload-cluster" namespace="beta" name="beta-workload-cluster" reconcileID="3a9e7730-e9aa-4c19-8397-2040 │ │ 1 cluster_accessor.go:257] "Connect failed" err="error creating HTTP client and mapper: cluster is not reachable: │ rate limiter Wait returned an error: context deadline exceeded - error from a previous attempt: EOF" controller="clustercache" contro │ controllerKind="Cluster" Cluster="beta/beta-workload-cluster" namespace="beta" name="beta-workload-cluster"  │ 7730-e9aa-4c19-8397-204032f2912a"                                                                                            │ 1 cluster_accessor.go:247] "Connecting" controller="clustercache" controllerGroup="cluster.x-k8s.io" controllerKi │ Cluster="beta/beta-workload-cluster" namespace="beta" name="beta-workload-cluster" reconcileID="52da4d29-b2d9-4dc3-9f16-7365 │ │ 1 cluster_accessor.go:257] "Connect failed" err="error creating HTTP client and mapper: cluster is not reachable: │ rate limiter Wait returned an error: context deadline exceeded - error from a previous attempt: EOF" controller="clustercache" contro │ controllerKind="Cluster" Cluster="beta/beta-workload-cluster" namespace="beta" name="beta-workload-cluster"  │ 4d29-b2d9-4dc3-9f16-736519ce9cb6"
                                    
                                        
                                            
                                            
                                                Jan 15
                                                '25  16:01
                                            
                                            
                                                oneironautjack


                                                                    
                                    
                                        
apiVersion: v1
data:
  aws-ebs-csi-external.yaml: |-
    apiVersion: v1
    kind: Secret
    metadata:
      name: aws-secret
      namespace: kube-system
    stringData:
      key_id: ""
      access_key: ""
    ---
    apiVersion: v1
    kind: ServiceAccount
    metadata:
      labels:
        app.kubernetes.io/name: aws-ebs-csi-driver
      name: ebs-csi-controller-sa
      namespace: kube-system
    ---
    apiVersion: v1
    kind: ServiceAccount
    metadata:
      labels:
        app.kubernetes.io/name: aws-ebs-csi-driver
      name: ebs-csi-node-sa
      namespace: kube-system
    ---
    apiVersion: rbac.authorization.k8s.io/v1
    kind: ClusterRole
    metadata:
      labels:
        app.kubernetes.io/name: aws-ebs-csi-driver
      name: ebs-external-attacher-role
    rules:
      - apiGroups:
          - ""
        resources:
          - persistentvolumes
        verbs:
          - get
          - list
          - watch
          - update
          - patch
      - apiGroups:
          - ""
        resources:
          - nodes
        verbs:
          - get
          - list
          - watch
      - apiGroups:
          - csi.storage.k8s.io
        resources:
          - csinodeinfos
        verbs:
          - get
          - list
          - watch
      - apiGroups:
          - storage.k8s.io
        resources:
          - volumeattachments
        verbs:
          - get
          - list
          - watch
          - update
          - patch
      - apiGroups:
          - storage.k8s.io
        resources:
          - volumeattachments/status
        verbs:
          - patch
    ---
    apiVersion: rbac.authorization.k8s.io/v1
    kind: ClusterRole
    metadata:
      labels:
        app.kubernetes.io/name: aws-ebs-csi-driver
      name: ebs-csi-node
    rules:
    - apiGroups:
      - ""
      resources:
      - podbservices
      verbs:
      - get
      - patch
    - apiGroups:
      - ""
      resources:
      - nodes
      verbs:
      - get
    ---
    apiVersion: rbac.authorization.k8s.io/v1
    kind: ClusterRole
    metadata:
      labels:
        app.kubernetes.io/name: aws-ebs-csi-driver
      name: ebs-external-provisioner-role
    rules:
      - apiGroups:
          - ""
        resources:
          - persistentvolumes
        verbs:
          - get
          - list
          - watch
          - create
          - delete
      - apiGroups:
          - ""
        resources:
          - persistentvolumeclaims
        verbs:
          - get
          - list
          - watch
          - update
      - apiGroups:
          - storage.k8s.io
        resources:
          - storageclasses
        verbs:
          - get
          - list
          - watch
      - apiGroups:
          - ""
        resources:
          - events
        verbs:
          - list
          - watch
          - create
          - update
          - patch
      - apiGroups:
          - snapshot.storage.k8s.io
        resources:
          - volumesnapshots
        verbs:
          - get
          - list
      - apiGroups:
          - snapshot.storage.k8s.io
        resources:
          - volumesnapshotcontents
        verbs:
          - get
          - list
      - apiGroups:
          - storage.k8s.io
        resources:
          - csinodes
        verbs:
          - get
          - list
          - watch
      - apiGroups:
          - ""
        resources:
          - nodes
        verbs:
          - get
          - list
          - watch
      - apiGroups:
          - coordination.k8s.io
        resources:
          - leases
        verbs:
          - get
          - watch
          - list
          - delete
          - update
          - create
      - apiGroups:
          - storage.k8s.io
        resources:
          - volumeattachments
        verbs:
          - get
          - list
          - watch
    ---
    apiVersion: rbac.authorization.k8s.io/v1
    kind: ClusterRole
    metadata:
      labels:
        app.kubernetes.io/name: aws-ebs-csi-driver
      name: ebs-external-resizer-role
    rules:
      - apiGroups:
          - ""
        resources:
          - persistentvolumes
        verbs:
          - get
          - list
          - watch
          - update
          - patch
      - apiGroups:
          - ""
        resources:
          - persistentvolumeclaims
        verbs:
          - get
          - list
          - watch
      - apiGroups:
          - ""
        resources:
          - persistentvolumeclaims/status
        verbs:
          - update
          - patch
      - apiGroups:
          - storage.k8s.io
        resources:
          - storageclasses
        verbs:
          - get
          - list
          - watch
      - apiGroups:
          - ""
        resources:
          - events
        verbs:
          - list
          - watch
          - create
          - update
          - patch
      - apiGroups:
          - ""
        resources:
          - podbservices
        verbs:
          - get
          - list
          - watch
    ---
    apiVersion: rbac.authorization.k8s.io/v1
    kind: ClusterRole
    metadata:
      labels:
        app.kubernetes.io/name: aws-ebs-csi-driver
      name: ebs-external-snapshotter-role
    rules:
      - apiGroups:
          - ""
        resources:
          - events
        verbs:
          - list
          - watch
          - create
          - update
          - patch
      - apiGroups:
          - ""
        resources:
          - secrets
        verbs:
          - get
          - list
      - apiGroups:
          - snapshot.storage.k8s.io
        resources:
          - volumesnapshotclasses
        verbs:
          - get
          - list
          - watch
      - apiGroups:
          - snapshot.storage.k8s.io
        resources:
          - volumesnapshotcontents
        verbs:
          - create
          - get
          - list
          - watch
          - update
          - delete
      - apiGroups:
          - snapshot.storage.k8s.io
        resources:
          - volumesnapshotcontents/status
        verbs:
          - update
    ---
    apiVersion: rbac.authorization.k8s.io/v1
    kind: ClusterRoleBinding
    metadata:
      labels:
        app.kubernetes.io/name: aws-ebs-csi-driver
      name: ebs-csi-attacher-binding
    roleRef:
      apiGroup: rbac.authorization.k8s.io
      kind: ClusterRole
      name: ebs-external-attacher-role
    subjects:
      - kind: ServiceAccount
        name: ebs-csi-controller-sa
        namespace: kube-system
    ---
    apiVersion: rbac.authorization.k8s.io/v1
    kind: ClusterRoleBinding
    metadata:
      labels:
        app.kubernetes.io/name: aws-ebs-csi-driver
      name: ebs-csi-provisioner-binding
    roleRef:
      apiGroup: rbac.authorization.k8s.io
      kind: ClusterRole
      name: ebs-external-provisioner-role
    subjects:
      - kind: ServiceAccount
        name: ebs-csi-controller-sa
        namespace: kube-system
    ---
    apiVersion: rbac.authorization.k8s.io/v1
    kind: ClusterRoleBinding
    metadata:
      labels:
        app.kubernetes.io/name: aws-ebs-csi-driver
      name: ebs-csi-resizer-binding
    roleRef:
      apiGroup: rbac.authorization.k8s.io
      kind: ClusterRole
      name: ebs-external-resizer-role
    subjects:
      - kind: ServiceAccount
        name: ebs-csi-controller-sa
        namespace: kube-system
    ---
    apiVersion: rbac.authorization.k8s.io/v1
    kind: ClusterRoleBinding
    metadata:
      labels:
        app.kubernetes.io/name: aws-ebs-csi-driver
      name: ebs-csi-snapshotter-binding
    roleRef:
      apiGroup: rbac.authorization.k8s.io
      kind: ClusterRole
      name: ebs-external-snapshotter-role
    subjects:
      - kind: ServiceAccount
        name: ebs-csi-controller-sa
        namespace: kube-system
    ---
    apiVersion: rbac.authorization.k8s.io/v1
    kind: ClusterRoleBinding
    metadata:
      labels:
        app.kubernetes.io/name: aws-ebs-csi-driver
      name: ebs-csi-node-binding
    roleRef:
      apiGroup: rbac.authorization.k8s.io
      kind: ClusterRole
      name: ebs-csi-node
    subjects:
    - kind: ServiceAccount
      name: ebs-csi-node-sa
      namespace: kube-system
    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      labels:
        app.kubernetes.io/name: aws-ebs-csi-driver
      name: ebs-csi-controller
      namespace: kube-system
    spec:
      replicas: 2
      selector:
        matchLabels:
          app: ebs-csi-controller
          app.kubernetes.io/name: aws-ebs-csi-driver
      template:
        metadata:
          labels:
            app: ebs-csi-controller
            app.kubernetes.io/name: aws-ebs-csi-driver
        spec:
          containers:
            - args:
                - --endpoint=$(CSI_ENDPOINT)
                - --logtostderr
                - --v=2
              env:
                - name: CSI_ENDPOINT
                  value: unix:///var/lib/csi/sockets/pluginproxy/csi.sock
                - name: CSI_NODE_NAME
                  valueFrom:
                    fieldRef:
                      fieldPath: spec.nodeName
                - name: AWS_ACCESS_KEY_ID
                  valueFrom:
                    secretKeyRef:
                      key: key_id
                      name: aws-secret
                      optional: true
                - name: AWS_SECRET_ACCESS_KEY
                  valueFrom:
                    secretKeyRef:
                      key: access_key
                      name: aws-secret
                      optional: true
              image: registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.25.0
              imagePullPolicy: IfNotPresent
              livenessProbe:
                failureThreshold: 5
                httpGet:
                  path: /healthz
                  port: healthz
                initialDelaySecondbservices: 10
                periodSecondbservices: 10
                timeoutSecondbservices: 3
              name: ebs-plugin
              ports:
                - containerPort: 9808
                  name: healthz
                  protocol: TCP
              readinessProbe:
                failureThreshold: 5
                httpGet:
                  path: /healthz
                  port: healthz
                initialDelaySecondbservices: 10
                periodSecondbservices: 10
                timeoutSecondbservices: 3
              volumeMounts:
                - mountPath: /var/lib/csi/sockets/pluginproxy/
                  name: socket-dir
            - args:
                - --csi-address=$(ADDRESS)
                - --v=2
                - --feature-gates=Topology=true
                - --extra-create-metadata
                - --leader-election=true
                - --default-fstype=ext4
              env:
                - name: ADDRESS
                  value: /var/lib/csi/sockets/pluginproxy/csi.sock
              image: registry.k8s.io/sig-storage/csi-provisioner:v3.6.2
              name: csi-provisioner
              volumeMounts:
                - mountPath: /var/lib/csi/sockets/pluginproxy/
                  name: socket-dir
            - args:
                - --csi-address=$(ADDRESS)
                - --v=2
                - --leader-election=true
              env:
                - name: ADDRESS
                  value: /var/lib/csi/sockets/pluginproxy/csi.sock
              image: registry.k8s.io/sig-storage/csi-attacher:v4.4.2
              name: csi-attacher
              volumeMounts:
                - mountPath: /var/lib/csi/sockets/pluginproxy/
                  name: socket-dir
            - args:
                - --csi-address=$(ADDRESS)
                - --leader-election=true
              env:
                - name: ADDRESS
                  value: /var/lib/csi/sockets/pluginproxy/csi.sock
              image: registry.k8s.io/sig-storage/csi-snapshotter:v6.3.2
              name: csi-snapshotter
              volumeMounts:
                - mountPath: /var/lib/csi/sockets/pluginproxy/
                  name: socket-dir
            - args:
                - --csi-address=$(ADDRESS)
                - --v=2
              env:
                - name: ADDRESS
                  value: /var/lib/csi/sockets/pluginproxy/csi.sock
              image: registry.k8s.io/sig-storage/csi-resizer:v1.9.2
              imagePullPolicy: Always
              name: csi-resizer
              volumeMounts:
                - mountPath: /var/lib/csi/sockets/pluginproxy/
                  name: socket-dir
            - args:
                - --csi-address=/csi/csi.sock
              image: registry.k8s.io/sig-storage/livenessprobe:v2.11.0
              name: liveness-probe
              volumeMounts:
                - mountPath: /csi
                  name: socket-dir
          nodeSelector:
            kubernetes.io/os: linux
          priorityClassName: system-cluster-critical
          serviceAccountName: ebs-csi-controller-sa
          tolerations:
            - key: CriticalAddonsOnly
              operator: Exists
            - effect: NoExecute
              operator: Exists
              tolerationSecondbservices: 300
            - key: node-role.kubernetes.io/master
              effect: NoSchedule
            - effect: NoSchedule
              key: node-role.kubernetes.io/control-plane
          affinity:
            nodeAffinity:
              requiredDuringSchedulingIgnoredDuringExecution:
                nodeSelectorTerms:
                  - matchExpressions:
                      - key: node-role.kubernetes.io/control-plane
                        operator: Exists
                  - matchExpressions:
                      - key: node-role.kubernetes.io/master
                        operator: Exists
          volumes:
            - emptyDir: {}
              name: socket-dir
    ---
    apiVersion: policy/v1
    kind: PodDisruptionBudget
    metadata:
      labels:
        app.kubernetes.io/name: aws-ebs-csi-driver
      name: ebs-csi-controller
      namespace: kube-system
    spec:
      maxUnavailable: 1
      selector:
        matchLabels:
          app: ebs-csi-controller
          app.kubernetes.io/name: aws-ebs-csi-driver
    ---
    apiVersion: apps/v1
    kind: DaemonSet
    metadata:
      labels:
        app.kubernetes.io/name: aws-ebs-csi-driver
      name: ebs-csi-node
      namespace: kube-system
    spec:
      selector:
        matchLabels:
          app: ebs-csi-node
          app.kubernetes.io/name: aws-ebs-csi-driver
      template:
        metadata:
          labels:
            app: ebs-csi-node
            app.kubernetes.io/name: aws-ebs-csi-driver
        spec:
          affinity:
            nodeAffinity:
              requiredDuringSchedulingIgnoredDuringExecution:
                nodeSelectorTerms:
                  - matchExpressions:
                      - key: eks.amazonaws.com/compute-type
                        operator: NotIn
                        values:
                          - fargate
          containers:
            - args:
                - node
                - --endpoint=$(CSI_ENDPOINT)
                - --logtostderr
                - --v=2
              env:
                - name: CSI_ENDPOINT
                  value: unix:/csi/csi.sock
                - name: CSI_NODE_NAME
                  valueFrom:
                    fieldRef:
                      fieldPath: spec.nodeName
              image: registry.k8s.io/provider-aws/aws-ebs-csi-driver:v1.25.0
              livenessProbe:
                failureThreshold: 5
                httpGet:
                  path: /healthz
                  port: healthz
                initialDelaySecondbservices: 10
                periodSecondbservices: 10
                timeoutSecondbservices: 3
              name: ebs-plugin
              ports:
                - containerPort: 9808
                  name: healthz
                  protocol: TCP
              securityContext:
                privileged: true
              volumeMounts:
                - mountPath: /var/lib/kubelet
                  mountPropagation: Bidirectional
                  name: kubelet-dir
                - mountPath: /csi
                  name: plugin-dir
                - mountPath: /dev
                  name: device-dir
            - args:
                - --csi-address=$(ADDRESS)
                - --kubelet-registration-path=$(DRIVER_REG_SOCK_PATH)
                - --v=2
              env:
                - name: ADDRESS
                  value: /csi/csi.sock
                - name: DRIVER_REG_SOCK_PATH
                  value: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
              image: registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.9.2
              name: node-driver-registrar
              volumeMounts:
                - mountPath: /csi
                  name: plugin-dir
                - mountPath: /registration
                  name: registration-dir
            - args:
                - --csi-address=/csi/csi.sock
              image: registry.k8s.io/sig-storage/livenessprobe:v2.11.0
              name: liveness-probe
              volumeMounts:
                - mountPath: /csi
                  name: plugin-dir
          nodeSelector:
            kubernetes.io/os: linux
          priorityClassName: system-node-critical
          serviceAccountName: ebs-csi-node-sa
          tolerations:
            - key: CriticalAddonsOnly
              operator: Exists
            - effect: NoExecute
              operator: Exists
              tolerationSecondbservices: 300
          volumes:
            - hostPath:
                path: /var/lib/kubelet
                type: Directory
              name: kubelet-dir
            - hostPath:
                path: /var/lib/kubelet/plugins/ebs.csi.aws.com/
                type: DirectoryOrCreate
              name: plugin-dir
            - hostPath:
                path: /var/lib/kubelet/plugins_registry/
                type: Directory
              name: registration-dir
            - hostPath:
                path: /dev
                type: Directory
              name: device-dir
      updateStrategy:
        rollingUpdate:
          maxUnavailable: 10%
        type: RollingUpdate
    ---
    apiVersion: storage.k8s.io/v1
    kind: CSIDriver
    metadata:
      labels:
        app.kubernetes.io/name: aws-ebs-csi-driver
      name: ebs.csi.aws.com
    spec:
      attachRequired: true
      podInfoOnMount: false
kind: ConfigMap
metadata:
  annotations:
    note: generated
  labels:
    app: clusterapi
    environment: beta
    type: generated
  name: beta-aws-ebs-csi-driver-addon
  namespace: beta
---
apiVersion: v1
data:
  aws-ccm-external.yaml: |
    ---
    apiVersion: apps/v1
    kind: DaemonSet
    metadata:
      name: aws-cloud-controller-manager
      namespace: kube-system
      labels:
        k8s-app: aws-cloud-controller-manager
    spec:
      selector:
        matchLabels:
          k8s-app: aws-cloud-controller-manager
      updateStrategy:
        type: RollingUpdate
      template:
        metadata:
          labels:
            k8s-app: aws-cloud-controller-manager
        spec:
          nodeSelector:
            node-role.kubernetes.io/control-plane: ""
          tolerations:
            - key: node.cloudprovider.kubernetes.io/uninitialized
              value: "true"
              effect: NoSchedule
            - key: node-role.kubernetes.io/control-plane
              effect: NoSchedule
          affinity:
            nodeAffinity:
              requiredDuringSchedulingIgnoredDuringExecution:
                nodeSelectorTerms:
                  - matchExpressions:
                      - key: node-role.kubernetes.io/control-plane
                        operator: Exists
          serviceAccountName: cloud-controller-manager
          containers:
            - name: aws-cloud-controller-manager
              image: registry.k8s.io/provider-aws/cloud-controller-manager:v1.28.3
              args:
                - --v=2
                - --cloud-provider=aws
                - --use-service-account-credentials=true
                - --configure-cloud-routes=false
              resources:
                requests:
                  cpu: 200m
          hostNetwork: true
    ---
    apiVersion: v1
    kind: ServiceAccount
    metadata:
      name: cloud-controller-manager
      namespace: kube-system
    ---
    apiVersion: rbac.authorization.k8s.io/v1
    kind: RoleBinding
    metadata:
      name: cloud-controller-manager:apiserver-authentication-reader
      namespace: kube-system
    roleRef:
      apiGroup: rbac.authorization.k8s.io
      kind: Role
      name: extension-apiserver-authentication-reader
    subjects:
      - apiGroup: ""
        kind: ServiceAccount
        name: cloud-controller-manager
        namespace: kube-system
    ---
    apiVersion: rbac.authorization.k8s.io/v1
    kind: ClusterRole
    metadata:
      name: system:cloud-controller-manager
    rules:
    - apiGroups:
      - ""
      resources:
      - events
      verbs:
      - create
      - patch
      - update
    - apiGroups:
      - ""
      resources:
      - nodes
      verbs:
      - '*'
    - apiGroups:
      - ""
      resources:
      - nodes/status
      verbs:
      - patch
    - apiGroups:
      - ""
      resources:
      - services
      verbs:
      - list
      - patch
      - update
      - watch
    - apiGroups:
      - ""
      resources:
      - services/status
      verbs:
      - list
      - patch
      - update
      - watch
    - apiGroups:
      - ""
      resources:
      - serviceaccounts
      verbs:
      - create
      - get
      - list
      - watch
    - apiGroups:
      - ""
      resources:
      - persistentvolumes
      verbs:
      - get
      - list
      - update
      - watch
    - apiGroups:
      - ""
      resources:
      - configmaps
      verbs:
      - list
      - watch
    - apiGroups:
      - ""
      resources:
      - endpoints
      verbs:
      - create
      - get
      - list
      - watch
      - update
    - apiGroups:
      - coordination.k8s.io
      resources:
      - leases
      verbs:
      - create
      - get
      - list
      - watch
      - update
    - apiGroups:
      - ""
      resources:
      - serviceaccounts/token
      verbs:
      - create
    ---
    kind: ClusterRoleBinding
    apiVersion: rbac.authorization.k8s.io/v1
    metadata:
      name: system:cloud-controller-manager
    roleRef:
      apiGroup: rbac.authorization.k8s.io
      kind: ClusterRole
      name: system:cloud-controller-manager
    subjects:
      - apiGroup: ""
        kind: ServiceAccount
        name: cloud-controller-manager
        namespace: kube-system
kind: ConfigMap
metadata:
  annotations:
    note: generated
  labels:
    app: clusterapi
    environment: beta
    type: generated
  name: beta-cloud-controller-manager-addon
  namespace: beta
---
apiVersion: addons.cluster.x-k8s.io/v1beta1
kind: ClusterResourceSet
metadata:
  labels:
    app: clusterapi
    environment: beta
  name: beta-crs-ccm
  namespace: beta
spec:
  clusterSelector:
    matchLabels:
      ccm: external
  resources:
  - kind: ConfigMap
    name: cloud-controller-manager-addon
  strategy: ApplyOnce
---
apiVersion: addons.cluster.x-k8s.io/v1beta1
kind: ClusterResourceSet
metadata:
  labels:
    app: clusterapi
    environment: beta
  name: beta-crs-csi
  namespace: beta
spec:
  clusterSelector:
    matchLabels:
      csi: external
  resources:
  - kind: ConfigMap
    name: aws-ebs-csi-driver-addon
  strategy: ApplyOnce
---
apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
kind: KubeadmConfigTemplate
metadata:
  labels:
    app: clusterapi
    environment: beta
  name: beta-worker-kubeadm-config-template
  namespace: beta
spec:
  template:
    spec:
      joinConfiguration:
        discovery:
          bootstrapToken:
            apiServerEndpoint: ""
        nodeRegistration:
          criSocket: /var/run/containerd/containerd.sock
          kubeletExtraArgs:
            cgroup-driver: systemd
            cloud-provider: aws
            container-log-max-files: "2"
            container-log-max-size: 100Mi
            event-record-qps: "10"
            protect-kernel-defaults: "true"
            tls-cipher-suites: TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,
              TLS_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,
              TLS_AES_256_GCM_SHA384
            tls-min-version: VersionTLS12
---
apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  labels:
    app: clusterapi
    environment: beta
  name: beta-workload-cluster
  namespace: beta
spec:
  clusterNetwork:
    podbservices:
      cidrBlocks:
      - 10.244.0.0/16
    services:
      cidrBlocks:
      - 10.100.0.0/16
  controlPlaneEndpoint:
    host: ""
    port: 6443
  controlPlaneRef:
    apiVersion: controlplane.cluster.x-k8s.io/v1beta1
    kind: KubeadmControlPlane
    name: beta-control-plane
  infrastructureRef:
    apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
    kind: AWSCluster
    name: beta-aws-workload-cluster
---
apiVersion: cluster.x-k8s.io/v1beta1
kind: MachineDeployment
metadata:
  labels:
    app: clusterapi
    environment: beta
  name: beta-dbbases-deployment
  namespace: beta
spec:
  clusterName: beta-workload-cluster
  replicas: 1
  selector:
    matchLabels:
      cluster.x-k8s.io/cluster-name: beta-workload-cluster
      cluster.x-k8s.io/role: worker
  template:
    metadata:
      labels:
        cluster.x-k8s.io/cluster-name: beta-workload-cluster
        cluster.x-k8s.io/role: worker
    spec:
      bootstrap:
        configRef:
          apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
          kind: KubeadmConfigTemplate
          name: beta-worker-kubeadm-config-template
      clusterName: beta-workload-cluster
      failureDomain: us-west-XX
      infrastructureRef:
        apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
        kind: AWSMachineTemplate
        name: beta-dbbases-worker-template
      version: v1.30.5
---
apiVersion: cluster.x-k8s.io/v1beta1
kind: MachineDeployment
metadata:
  labels:
    app: clusterapi
    environment: beta
  name: beta-dbservices-deployment
  namespace: beta
spec:
  clusterName: beta-workload-cluster
  replicas: 1
  selector:
    matchLabels:
      cluster.x-k8s.io/cluster-name: beta-workload-cluster
      cluster.x-k8s.io/role: worker
  template:
    metadata:
      labels:
        cluster.x-k8s.io/cluster-name: beta-workload-cluster
        cluster.x-k8s.io/role: worker
    spec:
      bootstrap:
        configRef:
          apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
          kind: KubeadmConfigTemplate
          name: beta-worker-kubeadm-config-template
      clusterName: beta-workload-cluster
      failureDomain: us-west-XX
      infrastructureRef:
        apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
        kind: AWSMachineTemplate
        name: beta-dbservices-worker-template
      version: v1.30.5
---
apiVersion: cluster.x-k8s.io/v1beta1
kind: MachineDeployment
metadata:
  labels:
    app: clusterapi
    environment: beta
  name: beta-labservices-deployment
  namespace: beta
spec:
  clusterName: beta-workload-cluster
  replicas: 1
  selector:
    matchLabels:
      cluster.x-k8s.io/cluster-name: beta-workload-cluster
      cluster.x-k8s.io/role: worker
  template:
    metadata:
      labels:
        cluster.x-k8s.io/cluster-name: beta-workload-cluster
        cluster.x-k8s.io/role: worker
    spec:
      bootstrap:
        configRef:
          apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
          kind: KubeadmConfigTemplate
          name: beta-worker-kubeadm-config-template
      clusterName: beta-workload-cluster
      failureDomain: us-west-XX
      infrastructureRef:
        apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
        kind: AWSMachineTemplate
        name: beta-labservices-worker-template
      version: v1.30.5
---
apiVersion: cluster.x-k8s.io/v1beta1
kind: MachineDeployment
metadata:
  labels:
    app: clusterapi
    environment: beta
  name: beta-performance-gpu-deployment
  namespace: beta
spec:
  clusterName: beta-workload-cluster
  replicas: 1
  selector:
    matchLabels:
      cluster.x-k8s.io/cluster-name: beta-workload-cluster
      cluster.x-k8s.io/role: worker
  template:
    metadata:
      labels:
        cluster.x-k8s.io/cluster-name: beta-workload-cluster
        cluster.x-k8s.io/role: worker
    spec:
      bootstrap:
        configRef:
          apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
          kind: KubeadmConfigTemplate
          name: beta-worker-kubeadm-config-template
      clusterName: beta-workload-cluster
      failureDomain: us-west-XX
      infrastructureRef:
        apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
        kind: AWSMachineTemplate
        name: beta-performance-gpu-worker-template
      version: v1.30.5
---
apiVersion: cluster.x-k8s.io/v1beta1
kind: MachineDeployment
metadata:
  labels:
    app: clusterapi
    environment: beta
  name: beta-worker-deployment
  namespace: beta
spec:
  clusterName: beta-workload-cluster
  replicas: 1
  selector:
    matchLabels:
      cluster.x-k8s.io/cluster-name: beta-workload-cluster
      cluster.x-k8s.io/role: worker
  template:
    metadata:
      labels:
        cluster.x-k8s.io/cluster-name: beta-workload-cluster
        cluster.x-k8s.io/role: worker
    spec:
      bootstrap:
        configRef:
          apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
          kind: KubeadmConfigTemplate
          name: beta-worker-kubeadm-config-template
      clusterName: beta-workload-cluster
      failureDomain: us-west-XX
      infrastructureRef:
        apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
        kind: AWSMachineTemplate
        name: beta-worker-worker-template
      version: v1.30.5
---
apiVersion: controlplane.cluster.x-k8s.io/v1beta1
kind: KubeadmControlPlane
metadata:
  annotations:
    machineset.cluster.x-k8s.io/skip-preflight-checks: All
  labels:
    app: clusterapi
    environment: beta
  name: beta-control-plane
  namespace: beta
spec:
  kubeadmConfigSpec:
    joinConfiguration:
      nodeRegistration:
        criSocket: /var/run/containerd/containerd.sock
        kubeletExtraArgs:
          cgroup-driver: systemd
          cloud-provider: aws
          container-log-max-files: "2"
          container-log-max-size: 100Mi
          event-record-qps: "10"
          protect-kernel-defaults: "true"
          tls-cipher-suites: TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,
            TLS_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,
            TLS_AES_256_GCM_SHA384
          tls-min-version: VersionTLS12
  machineTemplate:
    infrastructureRef:
      apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
      kind: AWSMachineTemplate
      name: beta-aws-control-plane-machine-template
  replicas: 3
  version: v1.30.5
---
apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
kind: AWSCluster
metadata:
  labels:
    app: clusterapi
    environment: beta
  name: beta-aws-workload-cluster
  namespace: beta
spec:
  bastion:
    enabled: false
  controlPlaneLoadBalancer:
    scheme: internal
    subnets:
    - subnet-XXXXXXXXXX54920
  network:
    securityGroupOverrides:
      apiserver-lb: sg-XXXXXXXXX4c03
      controlplane: sg-XXXXXXXXX4c03
      lb: sg-XXXXXXXXXX4c03
      node: sg-XXXXXXXXXX4c03
    subnets:
    - id: subnet-XXXXXXXXXX54920
    vpc:
      id: vpc-XXXXXXXXXXXXX
  region: us-west-1
  sshKeyName: XXXXXXXXX
---
apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
kind: AWSMachineTemplate
metadata:
  annotations:
    machineset.cluster.x-k8s.io/skip-preflight-checks: All
  labels:
    app: clusterapi
    environment: beta
  name: beta-aws-control-plane-machine-template
  namespace: beta
spec:
  template:
    spec:
      ami:
        id: ami-XXXXXXXXXX
      iamInstanceProfile: my-instance-profile
      instanceType: t3.2xlarge
      rootVolume:
        size: 400
        type: gp3
      sshKeyName: my-key
      subnet:
        id: subnet-XXXXXXXXXX54920
---
apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
kind: AWSMachineTemplate
metadata:
  labels:
    app: clusterapi
    environment: beta
  name: beta-dbbases-worker-template
  namespace: beta
spec:
  template:
    spec:
      additionalTags:
        Name: database
        NodeType: worker
        provisioner: clusterapi
      ami:
        id: XXXXXXXXXX
      iamInstanceProfile: my-instance-profile
      instanceType: t3.2xlarge
      rootVolume:
        size: 400
        type: gp3
      sshKeyName: my-key
---
apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
kind: AWSMachineTemplate
metadata:
  labels:
    app: clusterapi
    environment: beta
  name: beta-dbservices-worker-template
  namespace: beta
spec:
  template:
    spec:
      additionalTags:
        NodeType: worker
        noshutdown: "true"
        provisioner: clusterapi
      ami:
        id: ami-XXXXXXXXXX
      iamInstanceProfile: my-instance-profile
      instanceType: t3.2xlarge
      rootVolume:
        size: 400
        type: gp3
      sshKeyName: my-key
---
apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
kind: AWSMachineTemplate
metadata:
  labels:
    app: clusterapi
    environment: beta
  name: beta-labservices-worker-template
  namespace: beta
spec:
  template:
    spec:
      additionalTags:
        noshutdown: "true"
        provisioner: clusterapi
      ami:
        id: ami-XXXXXXXXXX
      iamInstanceProfile: my-instance-profile
      instanceType: t3.2xlarge
      rootVolume:
        size: 400
        type: gp3
      sshKeyName: my-key
---
apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
kind: AWSMachineTemplate
metadata:
  labels:
    app: clusterapi
    environment: beta
  name: beta-performance-gpu-worker-template
  namespace: beta
spec:
  template:
    spec:
      additionalTags:
        noshutdown: "true"
        provisioner: clusterapi
      ami:
        id: ami-XXXXXXXXXX
      iamInstanceProfile: my-instance-profile
      instanceType: t3.2xlarge
      rootVolume:
        size: 400
        type: gp3
      sshKeyName: my-key
---
apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
kind: AWSMachineTemplate
metadata:
  labels:
    app: clusterapi
    environment: beta
  name: beta-worker-worker-template
  namespace: beta
spec:
  template:
    spec:
      additionalTags:
        noshutdown: "true"
        provisioner: clusterapi
      ami:
        id: ami-XXXXXXXXXX
      iamInstanceProfile: my-instance-profile
      instanceType: t3.2xlarge
      rootVolume:
        size: 400
        type: gp3
      sshKeyName: my-key


                                    
                                    
                                        
                                            
                                            
                                                Jan 15
                                                '25  16:01
                                            
                                            
                                                oneironautjack
                                            
                                        
                                    
                                    
                                
                                                                    
                                    
                                        
Additional info: Using Classic ELB. Could this be related? #5139

                                    
                                    
                                        
                                            
                                            
                                                Jan 20
                                                '25  17:01
                                            
                                            
                                                oneironautjack
                                            
                                        
                                    
                                    
                                
                                                                    
                                    
                                        
Yes I think it's related to this. We are actively working on a fix for v1.30 and higher.
Note: we are going to deprecate classic elb support.

                                    
                                    
                                        
                                            
                                            
                                                Feb 08
                                                '25  14:02
                                            
                                            
                                                richardcase
                                            
                                        
                                    
                                    
                                
                                                                    
                                    
                                        
I switched to NLB and the load balancer completes but the control plane instances still fail to spin up.  This is a "bring your own VPC" use case.

                                    
                                    
                                        
                                            
                                            
                                                Feb 10
                                                '25  18:02
                                            
                                            
                                                oneironautjack
                                            
                                        
                                    
                                    
                                
                                                                    
                                    
                                        
Any other ideas why the EC2 instances fail to spawn? I don't even see an API RunInstances request in CloudTrail. It still complains about waiting on preflight tests to pass.

                                    
                                    
                                        
                                            
                                            
                                                Feb 11
                                                '25  15:02
                                            
                                            
                                                oneironautjack
                                            
                                        
                                    
                                    
                                
                                                                    
                                    
                                        
https://github.com/kubernetes-sigs/cluster-api-provider-aws/pull/5209

                                    
                                    
                                        
                                            
                                            
                                                Feb 18
                                                '25  13:02
                                            
                                            
                                                oneironautjack
                                            
                                        
                                    
                                    
                                
                                                                    
                                    
                                        
@oneironautjack Did you determine a root cause or fix? I'm having this exact issue with Openstack instances not being created despite the Openstack load balancer getting created. I'm using clusterctl v1.9.6

                                    
                                    
                                        
                                            
                                            
                                                Mar 31
                                                '25  19:03
                                            
                                            
                                                phil-bladen
                                            
                                        
                                    
                                    
                                
                                                                    
                                    
                                        

@oneironautjack Did you determine a root cause or fix? I'm having this exact issue with Openstack instances not being created despite the Openstack load balancer getting created. I'm using clusterctl v1.9.6

Unfortunately, no. I can't seem to get any traction on the issue. It seems the focus is on Managed(EKS) clusters and not BYO environments. Please keep me in the loop if you find any solutions and I'll do the same. Thanks!

                                    
                                    
                                        
                                            
                                            
                                                Apr 15
                                                '25  18:04
                                            
                                            
                                                oneironautjack
                                            
                                        
                                    
                                    
                                
                                                                    
                                    
                                        
Try to connect to the instance by ssh. Usually there're some problems with kubelet and it does not starts on the port that NLB checks in target group (6443).

                                    
                                    
                                        
                                            
                                            
                                                May 23
                                                '25  10:05
                                            
                                            
                                                abrekhov
                                            
                                        
                                    
                                    
                                
                                                                    
                                    
                                        

Try to connect to the instance by ssh. Usually there're some problems with kubelet and it does not starts on the port that NLB checks in target group (6443).

Its been a while since I've revisited this PoC but if I recall, I saw no clue that the EC2 instance was even being created

                                    
                                    
                                        
                                            
                                            
                                                Aug 13
                                                '25  17:08
                                            
                                            
                                                oneironautjack
                                            
                                        
                                    
                                    
                                
                                                                    
                                    
                                        
The Kubernetes project currently lacks enough contributors to adequately respond to all issues.
This bot triages un-triaged issues according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue as fresh with /remove-lifecycle stale
Close this issue with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale

                                    
                                    
                                        
                                            
                                            
                                                Nov 11
                                                '25  17:11
                                            
                                            
                                                k8s-triage-robot

cluster-api-provider-aws cluster-api-provider-aws copied to clipboard

Load balancer is created but EC2 instances are not created

cluster-api-provider-aws
cluster-api-provider-aws copied to clipboard