Adding NVMeDiskPlacement for diffdiskplacement field for v6 VMs in azcli

[geraldo29]

Preconditions

• [X] No need to upgrade Python SDK or the Python SDK is ready.

Related command

az vm create

Resource Provider

Microsoft.Compute

Description of Feature or Work Requested

Business Impact: 6.5% of IaaS VMs use ephemeral OS disk. Ephemeral OS disk is considered necessary for customers such as AKS who wish to adopt v6 SKUs. Today, Ephemeral OS disks support CacheDisk and ResourceDisk placement types. NVMe local disks are currently not supported as a valid placement type to customers. We are planning to release the support for v6 VMs in Ge. We need support from the Azure Portal team to expose a new NVMe placement type to allow ephemeral OS disk support for v6 SKUs with NVMe (MFND) disks. Tasks:

Document - ephemeral support in v6 SKUs.docx

Minimum API Version Required

2024-03-01

Swagger PR link / SDK link

https://github.com/Azure/azure-rest-api-specs/pull/27857

Request Example

https://github.com/Azure/azure-rest-api-specs/blob/main/specification/compute/resource-manager/Microsoft.Compute/ComputeRP/stable/2024-03-01/examples/virtualMachineExamples/VirtualMachine_Create_WithADiffOsDiskUsingDiffDiskPlacementAsNvmeDisk.json

Target Date

2024-06-01

PM Contact

@viveksigla

Engineer Contact

@glopezrosa

Additional context

Powershell github issue: https://github.com/Azure/azure-powershell-cmdlet-review-pr/issues/1431 Pull request https://msazure.visualstudio.com/One/_git/Compute-CPlat-Core/pullrequest/9515273?_a=files&path=/src/CRP/Dev/ComputeResourceProvider/Service/API/ApiVersioning.cs

Work Item: https://msazure.visualstudio.com/One/_workitems/edit/24833361

There are two feature flags and one region config setting.

V6 vm sizes have feature flags ex: Standard_D8ads_v6 is under feature flag Microsoft.Compute/DALDV6Series

Feature for enabling new Nvme Ephemeral preview features for a subscription. LocalNvmeEphemeralDiskPlacementPreview = "Microsoft.Compute/LocalNvmeEphemeralDiskPlacementPreview",

there is currently a Region dynamic config setting: EnableNvmeEphemeralDiskPlacement which is currently not being used in canary regions. Please let me know which regions you would like to enable and we can get that working.

For testing: For testing, we need to setup a v6 sku. If the sku does not support NVME, and the disk placement is passed as Nvme Disk, it will fail.

[yonzhan]

Thank you for opening this issue, we will look into it.

[yanzhudd]

Hi @geraldo29, it seems that our subId is not registered for this feature. Could you please help to register our subId (0b1f6471-1bf0-4dda-aec3-cb9272f09590)?

[yanzhudd]

Please let me check with you if the newly added option NvmeDisk need to be added for az vm update command as well?

[glopezrosa]

az vm update [--add] [--capacity-reservation-group] [--disk-caching] [--disk-controller-type {NVMe, SCSI}] [--enable-hibernation {false, true}] [--enable-proxy-agent {false, true}] [--enable-secure-boot {false, true}] [--enable-vtpm {false, true}] [--ephemeral-os-disk-placement {CacheDisk, ResourceDisk}]

         Yes, we need to add it as an option for ephemeral os disk placement. 
         
         Thanks for catching this. 

[glopezrosa]

Hi @geraldo29, it seems that our subId is not registered for this feature. Could you please help to register our subId (0b1f6471-1bf0-4dda-aec3-cb9272f09590)?

Done.

[glopezrosa]

What regions are you testing in? Currently only west europe is turned on and some canary regions. let me know if you face problems with the region.

[geraldo29]

@yanzhudd can we add VMSS changes here if not yet completed?

For CLI:

RequestExample: https://github.com/Azure/azure-rest-api-specs/blob/main/specification/compute/resource-manager/Microsoft.Compute/ComputeRP/stable/2024-03-01/examples/virtualMachineScaleSetExamples/VirtualMachineScaleSet_CreateA_WithDiffOsDiskUsingDiffDiskPlacementAsNvmeDisk.json

Add NVMeDisk as a valid option for parameter --ephemeral-os-disk-placement in the below az command az vmss create
--resource-group myResourceGroup
--name myVMss
--image imageName
--ephemeral-os-disk true
--ephemeral-os-disk-placement NvmeDisk \ <-- support NvmeDisk in this parameter --os-disk-caching ReadOnly
--admin-username azureuser
--generate-ssh-keys

[yanzhudd]

Hi @geraldo29, Okay, we'll update the vmss command as well. When we tested, the following error was reported. It seems that 'NvmeDisk' is not supported for VM size Standard_DS1_v2. Could you please share what VM size we can use?

[yanzhudd]

Hi @glopezrosa @geraldo29 could you please help with this issue https://github.com/Azure/azure-cli/issues/28837#issuecomment-2104096787? Please note that we will freeze the code at 05/14/2024 10:00 UTC for the upcoming release train. Thus, if you want this change being released in this sprint, please provide the information as soon as possible, otherwise it will have to be postponed to the next sprint.

[geraldo29]

Hey here is the vm to use. Only v6 vms can use NvmeDisk placement.

Standard_D8ads_v6 is under feature flag Microsoft.Compute/DALDV6Series. I have registered your subscription to this feature flag.

To get access we need to pin our subscription to this cluster: AMS25PrdApp73 in region west europe

Here is the template to use to pin your cluster: ICM pin cluster request: https://portal.microsofticm.com/imp/v3/incidents/create?tmpl=q1C3yo

and here is an example: https://portal.microsofticm.com/imp/v5/incidents/details/497549527/summary?tmpl=q1C3yo

I didnt create the request because the cluster may already be in production. but if not, please create the request to pin your sub to the cluster if possible.

[yanzhudd]

Hi @geraldo29 it seems that Standard_D8ads_v6 is not available for our subId. And this subId is not only used by us but also other team and users, so I think I don't have the access to pin it to a specific cluster. So, could you please help to verify if this private package meets your expectation? The related commands are az vm/vmss create/update.

Thanks in advance! Please note we will freeze code at 05/14/2024 10:00 UTC, please let us know asap if there is any issues so that we could modify it and catch the release train.