[SFMC Feature]: Allow mixed stateless node types of both spot- and non-spot VMs

[WhitWaldo]

Is your feature request related to a problem? Please describe your workload details. Spot VMs are significantly cheaper than non-spot VMs, but are still quite capable of doing stateless work.

The latest preview of the SF runtime and ARM deployments supports the designation of a node type as both stateless and isSpot or not. Managed SF instances deploy a new VMSS for each node type defined which would generally be fine as I can deploy resources constrained to run in either node type, but isn't ideal because of the limitations of other Azure services.

Namely, for non-path-based (basic) rules, Application Gateway (v1 and v2) only allows a single backend address pool per rule. As these backend address pools typically are 1:1 to VMSSes, it means that there is no scenario using App Gateway that I can receive requests on a domain and forward to one node type, but not two. That's not ideal because it means that App Gateway would have to exclusively use the non-spot VMs (since I need the guarantee of something being there to handle the request) and leaves the non-spot VMs significantly less versatile (especially to handling stateless web traffic load).

Describe the solution you'd like I would like to be able to indicate that a stateless node type enables a minimum number of non-spot VMs but autoscales up with spot-VMs if available and non-spot VMs if not available. This would allow me to configure App Gateway to forward requests directly to the singular node type for processing.

Describe alternatives you've considered The only workaround I see here is to set up a dedicated SF YARP app for each inbound web app that I want to expose and have it handle the load balancing of the requests to each the service instances (since it would know no placement constraints there).

But again, this is less than ideal as SF YARP currently only supports path-based routing and not say, domain-based, so I need a separate app created and configured for each and every domain endpoint which just adds to my solution complexity.

Should SF YARP receive support for domain-based routing, this would simplify this workaround, but it still means I'm shouldering load on my compute to run YARP where I'm paying for that utility and availability already through my App Gateway costs.

Area/Component: Node Types/Azure Infrastructure

Additional context N/A

Service Fabric Runtime Version: 9.*

Environment: Azure

---

Assignees: /cc @microsoft/service-fabric-triage