sdesai345

[Sandbox] Kubernetes AI Toolchain Operator (KAITO)

5

### Application contact emails [email protected], [email protected], [email protected], [email protected], [email protected] ### Project Summary KAITO automates the deployment of AI models and associated infrastructure provisioning on a Kubernetes cluster ### Project Description...

On January 10, 2025, the AKS GPU VHD Image (preview) will be retired. Follow the detailed steps in [our documentation](https://learn.microsoft.com/azure/aks/gpu-cluster?tabs=add-ubuntu-gpu-node-pool#use-the-aks-gpu-image-preview) create GPU-enabled node pools using the alternative supported options on...

[BUG] NVIDIA A10 GPU GRID driver licensing fix

3

It was discovered that Azure GPU VM sizes using 550 GRID driver (NVIDIA A10 series) installed by AKS on Ubuntu nodes, may be affected by license status issues caused by...

Support multi-node distributed inference

1

**Is your feature request related to a problem? Please describe.** To serve larger language models with billions of parameters, users want to deploy multi-node inference on their Kubernetes cluster using...

Expand this sub-project readme https://github.com/kaito-project/keda-kaito-scaler with 2-3 use cases and add to a new doc in the `Features` section of the KAITO website

Improve the BYO HuggingFace model experience

1

Improve the BYO Hugging Face model experience: leverage existing AIKit tools to build customized model images from HF, for Kaito workspace to deploy the customized image in single node inference...

docs: New conceptual guide for GAIE and KAITO

2

**Reason for Change**: New conceptual guide to add context and visual diagrams for ./gateway-api-inference-extension.md

Remove soon-to-retire VM sizes from Azure sku handler

2

Please update the [Azure sku handler](https://github.com/kaito-project/kaito/blob/8f6f312d4f6fe7ededd33b40025dcb5019c58060/pkg/sku/azure_sku_handler.go) and proactively remove the skus impacted by [NV-series retirement](https://learn.microsoft.com/en-us/azure/virtual-machines/sizes/retirement/nv-series-retirement) and [NCv3, NC24rs retirement](https://learn.microsoft.com/en-us/azure/virtual-machines/ncv3-nc24rs-retirement) later this year.

**Is your feature request related to a problem? Please describe.** Azure teams request for KAITO to validate and support node pool creation of Azure Linux 3.0 GPU nodes.