Update VMSS + LB to use NATv2

[fitzgeraldsteele]

Related command

az vmss

Resource Provider

Microsoft.compute

Description of Feature or Work Requested

With VMSS create there is currently client logic like:

• If VMSS Uniform, configure NAT Pool,
• If VMSS Flex, do not configure NAT pool

LB has recently launched an update to NAT rules that work for both VMSS Uniform and Flex. We would like to revise the VMSS create to always set up NAT rules via the new API:

https://azure.microsoft.com/en-in/blog/manage-port-forwarding-for-backend-pool-with-azure-load-balancer/

      {
        "name": "[concat(variables('lbName'), '/', 'natRule')]",
        "type": "Microsoft.Network/loadBalancers/inboundNatRules",
        "apiVersion": "2021-05-01",
	    "location":"[parameters('location')]",
	    "dependsOn":[
            "[concat('Microsoft.Network/loadBalancers/', variables('lbName'))]"
        ],
        "properties": {
      	  "frontendIPConfiguration": {
      	    "id": "[variables('frontendIPConfigID')]"
      	  },
      	  "protocol": "TCP",
      	  "frontendPortRangeStart": 50000,
      	  "frontendPortRangeEnd": 50100,
	  "backendPort": 3389,
      	  "idleTimeoutInMinutes": 4,
      	  "backendAddressPool": {
      	    "id": "[variables('backendaddressPoolID')]"
	    }
	  }

Minimum API Version Required

2022-03-01 compute 2021-05-01 network

Swagger Link

Normal CRP swagger Normal NRP swagger

Target Date

7/30/2022

[yonzhan]

Compute

[zhoxing-ms]

For creating Uniform VMSS, if users do not specify the --nat-pool-name, CLI will take the load balancer name plus the suffix of "NatPool" as the default name of nat pool. code link: https://github.com/Azure/azure-cli/blob/74e96090f3a4283522f4b2f70e971abfc02a74ca/src/azure-cli/azure/cli/command_modules/vm/custom.py#L3114-L3118 @fitzgeraldsteele For Flex VMSS, do we need to keep the same logic?

[fitzgeraldsteele]

Nat Pool is not available for VMSS Flex (since they are just regular VMs). Should fail deployment if Nat Pool and VMSS Flex are specified at the same time. That's why network team came up with Nat Rules V2, which works for VMSS Flex, Uniform and standard VMs.

The suggestion in this request was to replace Nat Pool with NAT Rules for both VMSS Uniform and Flex. Admittedly, I did not consider this case if the user specifies NAT pool.

Is there a way we can mat Nat Rules v2 the default, and allow customers to opt in to NAT Pool for VMSS Unform if they want?

Jerry Steele Microsoft senior program manager 425.421.2566

---

From: Xing Zhou @.> Sent: Wednesday, July 6, 2022 1:17 AM To: Azure/azure-cli @.> Cc: Jerry Steele @.>; Mention @.> Subject: Re: [Azure/azure-cli] Update VMSS + LB to use NATv2 (Issue #22682)

For creating Uniform VMSS, if users do not specify the --nat-pool-name, CLI will take the load balancer name plus the suffix of NatPool as the default name of nat pool. code link: https://github.com/Azure/azure-cli/blob/74e96090f3a4283522f4b2f70e971abfc02a74ca/src/azure-cli/azure/cli/command_modules/vm/custom.py#L3114-L3118https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAzure%2Fazure-cli%2Fblob%2F74e96090f3a4283522f4b2f70e971abfc02a74ca%2Fsrc%2Fazure-cli%2Fazure%2Fcli%2Fcommand_modules%2Fvm%2Fcustom.py%23L3114-L3118&data=05%7C01%7Cjerry.steele%40microsoft.com%7Ce49820ea9d054ce4b97c08da5f2805f7%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637926922751525360%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=fqq1Nw01dQSWp%2FGT3foCymnICFCD6HR2KQMd3MdAU18%3D&reserved=0 @fitzgeraldsteelehttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ffitzgeraldsteele&data=05%7C01%7Cjerry.steele%40microsoft.com%7Ce49820ea9d054ce4b97c08da5f2805f7%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637926922751525360%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=LbRDnBCxpHZ0bBAIkd3eoynIhRQloRIQxubi0yE3CVY%3D&reserved=0 For Flex VMSS, do we need to keep the same logic?

— Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAzure%2Fazure-cli%2Fissues%2F22682%23issuecomment-1175926077&data=05%7C01%7Cjerry.steele%40microsoft.com%7Ce49820ea9d054ce4b97c08da5f2805f7%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637926922751525360%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=l1C9NOzt%2BYve48ept6MZuovfInQA%2FwpisZ%2FE%2BbrJkVc%3D&reserved=0, or unsubscribehttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAABGQSGWBR3KFGJR3AHSBZ3VSU6LBANCNFSM5XOGLUNQ&data=05%7C01%7Cjerry.steele%40microsoft.com%7Ce49820ea9d054ce4b97c08da5f2805f7%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637926922751525360%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Vy%2F%2BVdKWTYLWtU8L7O7Zf5mWEDDycTBCEzSCqSRddwc%3D&reserved=0. You are receiving this because you were mentioned.Message ID: @.***>

[zhoxing-ms]

@fitzgeraldsteele Please note that if we directly replace the resources created by default from NAT Pool to Nat Rules V2 for az vmss create command, I am worried that this will cause breaking change to users, especially when NAT Pool and Nat Rules V2 have great differences in feature characteristics

Could we consider providing a new parameter --nat-rule to allow users to use Nat Rules V2 instead of NAT Pool, but do not modify the default behavior to avoid breaking change? What do you think of this solution?

[zhoxing-ms]

@fitzgeraldsteele Please note that since the current design has breaking change, so we can't directly implement it without full discussion. Please see if my suggestions above are acceptable and reply to me before Wednesday. Otherwise, if the problem is confirmed too late, this sprint will not have enough time for us to develop and test, so we will have to postpone it to the next sprint (09-06) Thanks for your understanding~

[fitzgeraldsteele]

Thanks, @Xing @.***>. For bringing this up and apologies for the delay.

@Load Balancer & NAT PM @.***> - can I get some help on this issue with CLI? Was working with Irene to update AzCLI VMSS create to use NAT v2. Is this a breaking change? What should the experience be for customers to use NAT pool vs Natv2?

Jerry Steele Microsoft senior program manager 425.421.2566

---

From: Xing Zhou @.> Sent: Monday, July 18, 2022 2:08 AM To: Azure/azure-cli @.> Cc: Jerry Steele @.>; Mention @.> Subject: Re: [Azure/azure-cli] Update VMSS + LB to use NATv2 (Issue #22682)

@fitzgeraldsteelehttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ffitzgeraldsteele&data=05%7C01%7Cjerry.steele%40microsoft.com%7C28c247410bea48da602b08da689d0cd7%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637937320988096431%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=JEMnPEkoVKK5n9w1IUvTovj6NYITJvZ25I%2FblbWPTNc%3D&reserved=0 Please note that since the current design has breaking change, so we can't directly implement it without full discussion. Please see if my suggestions above are acceptable and reply to me before Wednesday. Otherwise, if the problem is confirmed too late, this sprint will not have enough time for us to develop and test, so we will have to postpone it to the next sprint (09-06) Thanks for your understanding~

— Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAzure%2Fazure-cli%2Fissues%2F22682%23issuecomment-1186952036&data=05%7C01%7Cjerry.steele%40microsoft.com%7C28c247410bea48da602b08da689d0cd7%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637937320988096431%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=rIJn0HFSf8u%2F7oxwcBu88Qfk9MoYaeTDihlec9sC8ck%3D&reserved=0, or unsubscribehttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAABGQSFZH4XN5SLDUWJYBQ3VUUNH7ANCNFSM5XOGLUNQ&data=05%7C01%7Cjerry.steele%40microsoft.com%7C28c247410bea48da602b08da689d0cd7%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637937320988096431%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=5qXLxgS7sbn%2FfHiVfRY5EdK4LmSzyvVmWhZgf05WAvM%3D&reserved=0. You are receiving this because you were mentioned.Message ID: @.***>

[anavinahar]

This is not a breaking change. It is the same functionality but an easier way to enable @zhoxing-ms

[zhoxing-ms]

@anavinahar OK, thanks for your clarification. Since Nat Rules V2 and NAT Pool have exactly the same functionality, and replacing NAT Pool with Nat Rules V2 will not cause breaking change, so do we only need to support users to use Nat Rules V2 to create Uniform and Flex VMSS? @anavinahar @fitzgeraldsteele

In other ways, may I ask do users still have the need or scenario to use NAT Pool to create Uniform VMSS? If not, we can consider only replacing NAT Pool with Nat Rules V2 without adding additional parameters to support NAT Pool for Uniform VMSS? @fitzgeraldsteele

[zhoxing-ms]

@fitzgeraldsteele @anavinahar Since we will launch the release of this sprint next week, please answer these questions before tomorrow, otherwise we will not have enough time to develop and release this feature in this sprint, then we have to postpone it to the next sprint (09-06)

[anavinahar]

Yes both are the same functionality. We can replace with nat rules v2

For VMSS Uniform it is @fitzgeraldsteele's call.

[zhoxing-ms]

@fitzgeraldsteele) Because the modification of this feature is complex (I found that the structure and level of these two resources in ARM template are inconsistent), and it affects many usages and tests. Since this sprint has less than five working days left, we don't have enough time to develop and test it comprehensively in this sprint. In order to avoid introducing unexpected risks and problems due to insufficient testing time, we plan to postpone it to the next sprint (09-06). Do you think it is acceptable?

[zhoxing-ms]

@fitzgeraldsteele @anavinahar @mahipdeora I have three more questions that need your help take a look~

1. Do we need to add a new parameter to support users to specify the name of NAT rule v2?
2. When the users specify the --nat-pool-name parameter, in order to avoid breaking changes, we still create NAT pool instead of NAT rule v2. In this case, do we need to print the warning log to remind users that the NAT pool is no longer recommended?
3. The az vmss list-instance-connection-info command will no longer work for Uniform VMSS which uses the NAT rule v2, because we can't get the backendIPConfiguration from inboundNatRules of loadBalancers anymore. REST service: load-balancers/get code link: https://github.com/Azure/azure-cli/blob/ffbcfd0526cfdbc124272cedb2066b2b4119471e/src/azure-cli/azure/cli/command_modules/vm/custom.py#L3530-L3534 May I ask is this problem is in line with your expectations? Or do we have any other way to get the value of backendIPConfiguration from server side?

[zhoxing-ms]

The az vmss list-connection-info command will no longer work for Uniform VMSS which uses the NAT rule v2, because we can't get the backendIPConfiguration from inboundNatRules of loadBalancers anymore.

@mahipdeora @anavinahar @fitzgeraldsteele These two points have been implemented in draft PR. Do you have any ideas or suggestions on the third question?

In addition, can we hide the --nat-pool-name parameter in the help information? Users can continue to use it to avoid breaking change, but they will get the following prompt when using it: Does this meet your expectations?