[Feature] Support custom images for nodes in AKS managed clusters

[nclaeys]

Is your feature request related to a problem? Please describe. It would be very useful for us if we could customize the images used by the AKS nodes, as already exists on EKS or GKE. We have 2 main usecases for this:

• packaging pre pulled images for our daemonsets/common used images and thus speeding up the startup of a node significantly
• building a custom image with sysbox installed, again to drastically speeding up the startup of these nodes (on eks: 8min -> 3min). Now we need to depend on the sysbox daemonset to configure the node but this is really slow when dynamically scaling nodes.

Describe the solution you'd like Allow nodepools to specify a custom image instead of only supporting the aks images. Open source the building of the aks images such that people can extend them.

Describe alternatives you've considered We looked at the CAPI project but do not want to create non-managed clusters. Other alternatives are hard to come by as the AKS nodes are quite closed source and it is very difficult to figure out how to build an Azure image as well as there is no way to configure custom images in nodepools.

[esolace88]

I agree with this feature add, and if possible, allow incorporating the Azure STIG Templates for Linux - Preview https://learn.microsoft.com/en-us/azure/azure-government/documentation-government-stig-linux-vm. Thanks!

[Dhruvin4530]

But how to use this in AKS?

[esolace88]

Hmm good question, maybe via az Cli preview extension like adding an additional option. For customer image maybe accept a VHD for along with OS.SKU flag. For STIG OS allow the STIG template to be used on existing OS.SKU options.

[--os-sku {AzureLinux, CBLMariner, Mariner, Ubuntu}]

AZ Preview https://learn.microsoft.com/en-us/cli/azure/aks?view=azure-cli-latest#az-aks-create(aks-preview)

STIG VM Creation https://learn.microsoft.com/en-us/azure/azure-government/documentation-government-stig-linux-vm

[Dhruvin4530]

I checked the docs and terraform official AKS module but it is not mentioned anything about custom image 😢

[mdulgheru]

"packaging pre pulled images for our daemonsets/common used images and thus speeding up the startup of a node significantly" This would be a great feature.

[ajinkya2801]

Dear Team, I would like to suggest a potential enhancement for Azure Kubernetes Service (AKS) node pool families: the ability to use custom node images. This feature would be a significant improvement, and I am eager to see it implemented by Microsoft. I am formally requesting consideration for this enhancement to be added to the AKS node pool roadmap.

[ajinkya2801]

Hi Team,

We are looking for a solution where we would like to have preconfigured nodes which has pre-fetch ACR images and we would like to create pool of such nodes so that when we execute our workflow containing pods of pre-fetched images does not take much time to deploy and get into running state.

As per current configuration, we have azure managed AKS cluster which create node based on load and deploy pods on it. Hence, this required additional time for pod to get into running state. To avoid this delay, we are looking for above mentioned solution.

[meiswjn]

I would also be interested in this feature for the reason of adding pre-loaded images and switching the container runtime to Sysbox. Artifact Streaming's "15% speed improvement" seem not that much compared to this option.

[shreyansh2953]

we are using aks managed cluster, as of today is the custom / hardened image feature for aks nodepools is available ?

[sjwaight]

Duplicate of #4110.

[microsoft-github-policy-service[bot]]

This issue has been marked as duplicate and has not had any activity for 1 day. It will be closed for housekeeping purposes.