eksctl
eksctl copied to clipboard
[Feature] Support KubeletExtraConfig for Windows nodegroups
When trying to create a Windows nodegroup, I got the error:
Error: couldn't create cluster provider from options: kubeletExtraConfig is not supported for WindowsServer2019FullContainer nodegroups (path=nodeGroups[0].kubeletExtraConfig)
However, the aws-eks-best-practices documentation specifically calls out the need to do this to get around the lack of an out-of-memory process killer in Windows itself:
If eksctl is the deployment tool, check the following documentation to customize the kubelet configuration https://eksctl.io/usage/customizing-the-kubelet/
The CloudFormation template referenced in that documentation takes the kubelet arguments as a parameter named BootstrapArguments and injects them into a custom bootstrap script written in Powershell:
<powershell>
[string]$EKSBinDir = "$env:ProgramFiles\Amazon\EKS"
[string]$EKSBootstrapScriptName = 'Start-EKSBootstrap.ps1'
[string]$EKSBootstrapScriptFile = "$EKSBinDir\$EKSBootstrapScriptName"
[string]$cfn_signal = "$env:ProgramFiles\Amazon\cfn-bootstrap\cfn-signal.exe"
& $EKSBootstrapScriptFile -EKSClusterName ${ClusterName} ${BootstrapArguments} 3>&1 4>&1 5>&1 6>&1
$LastError = if ($?) { 0 } else { $Error[0].Exception.HResult }
& $cfn_signal --exit-code=$LastError --stack="${AWS::StackName}" --resource="NodeGroup" --region=${AWS::Region}
</powershell>
In order to set up the nodegroup with these memory settings, we have to be able to specify kubeletExtraConfig with Windows nodes. The specific arguments they call out would look like this:
kubeletExtraConfig:
kubeReserved:
memory: "0.5Gi"
ephemeral-storage: "1Gi"
systemReserved:
memory: "1.5Gi"
ephemeral-storage: "1Gi"
evictionHard:
memory.available: "200Mi"
nodefs.available: "10%"