kuma icon indicating copy to clipboard operation
kuma copied to clipboard

Proper support for locality weighted load balancing

Open lobkovilya opened this issue 2 years ago β€’ 11 comments

Summary

Today locality aware load balancing is implemented using priorities - local=0, remote=1. We couldn't add Locality weighted load balancing because it's not compatible with lb subsetting that was used for TrafficRoutes. Since we don't use subsetting anymore (every TrafficRoute.Split is a separate cluster) probably we can properly support locality weighted load balancing.

This issue aims to track the progress of investigation around the possibility to support locality weighted load balancing.

lobkovilya avatar Aug 25 '21 19:08 lobkovilya

πŸ‘πŸ» πŸ‘€

karlhaworth avatar Sep 12 '21 03:09 karlhaworth

This issue was inactive for 30 days it will be reviewed in the next triage meeting and might be closed. If you think this issue is still relevant please comment on it promptly or attend the next triage meeting.

github-actions[bot] avatar Dec 24 '21 08:12 github-actions[bot]

This needs to be implemented with multiple layers in mind. For example including k8s: https://kubernetes.io/docs/concepts/services-networking/topology-aware-hints/ Ideally if we have topology.kubernetes.io/zone and multiple Kuma zones it should be possible to have 3 priorities:

  1. same k8s zone and same Kuma zone
  2. different k8s zone but same Kuma zone
  3. different Kuma zone

The traffic would progressively migrate from the closest network to the furthest as the service degrades.

This should be potentially configurable on a per route basis as a list of tags .e.g:

type: TrafficRoute
name: route-all-default
mesh: default
sources:
  - match:
      kuma.io/service: '*'
destinations:
  - match:
      kuma.io/service: '*'
conf:
  localityAwareLoadBalancingPriorities: [topology.kubernetes.io/zone, kuma.io/zone] # If this is unset traffic will remain stuck on the local zone?

lahabana avatar Feb 09 '22 09:02 lahabana

xref #3456 which is probably going to be implemented at the same time.

lahabana avatar Feb 09 '22 09:02 lahabana

Triage: This might also mean that standalone can have localityAwareLoadBalancing too.

lahabana avatar Feb 14 '22 15:02 lahabana

Note: This should also include exposing the overprovisioning_factor value

michaelbeaumont avatar Feb 18 '22 12:02 michaelbeaumont

This issue was inactive for 30 days it will be reviewed in the next triage meeting and might be closed. If you think this issue is still relevant please comment on it promptly or attend the next triage meeting.

github-actions[bot] avatar Mar 21 '22 08:03 github-actions[bot]

This issue was inactive for 30 days it will be reviewed in the next triage meeting and might be closed. If you think this issue is still relevant please comment on it promptly or attend the next triage meeting.

github-actions[bot] avatar Apr 21 '22 08:04 github-actions[bot]

This issue was inactive for 30 days it will be reviewed in the next triage meeting and might be closed. If you think this issue is still relevant please comment on it promptly or attend the next triage meeting.

github-actions[bot] avatar May 26 '22 08:05 github-actions[bot]

This issue was inactive for 30 days it will be reviewed in the next triage meeting and might be closed. If you think this issue is still relevant please comment on it promptly or attend the next triage meeting.

github-actions[bot] avatar Jun 27 '22 08:06 github-actions[bot]

This issue was inactive for 30 days it will be reviewed in the next triage meeting and might be closed. If you think this issue is still relevant please comment on it promptly or attend the next triage meeting.

github-actions[bot] avatar Jul 28 '22 08:07 github-actions[bot]

This issue was inactive for 90 days. It will be reviewed in the next triage meeting and might be closed. If you think this issue is still relevant, please comment on it or attend the next triage meeting.

github-actions[bot] avatar Oct 27 '22 08:10 github-actions[bot]

This issue was inactive for 90 days. It will be reviewed in the next triage meeting and might be closed. If you think this issue is still relevant, please comment on it or attend the next triage meeting.

github-actions[bot] avatar Jan 26 '23 08:01 github-actions[bot]

This issue was inactive for 90 days. It will be reviewed in the next triage meeting and might be closed. If you think this issue is still relevant, please comment on it or attend the next triage meeting.

github-actions[bot] avatar Apr 27 '23 07:04 github-actions[bot]

Potentially related upstream k8s work:

  • https://github.com/kubernetes/kubernetes/pull/116522
  • https://github.com/kubernetes/enhancements/issues/2433

johnharris85 avatar May 25 '23 21:05 johnharris85

This issue was inactive for 90 days. It will be reviewed in the next triage meeting and might be closed. If you think this issue is still relevant, please comment on it or attend the next triage meeting.

github-actions[bot] avatar Aug 24 '23 07:08 github-actions[bot]

In a quick brainstorm with @johnharris85 we came up with something like:

targetRef:
  kind: Mesh
to:
  targetRef:
    kind: MeshService
    name: foo
  defaults:
    localityAware:
      - tag: k8s/node
        type: 'self'
      - tag: k8s/az
        type: 'self'
      - tag: kuma.io/zone
        type: 'affinity'
         groups:
            - ['eu-east', 'eu-west']
            - ['us-east', 'us-west']
         fallback: 'any'
      - tag: kuma.io/zone
        type: 'affinity'
         groups:
            - ['eu-east', 'eu-west', 'eu-central']
            - ['us-east', 'us-west', 'us-central']
         fallback: 'fail'

We really need to first enumerate all the use-cases we want to deal with.

lahabana avatar Sep 26 '23 07:09 lahabana

Some useful envoy stuff:

https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/upstream/aggregate_cluster https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/upstream/load_balancing/priority#arch-overview-load-balancing-priority-levels

lahabana avatar Sep 26 '23 12:09 lahabana

Use cases:

  • loadbalance within a zone to favour traffic on the same node, on the same AZ, on the same rack (for universal)
  • disable disabling all cross zone traffic for a service
  • only failover to one zone from another (grouped affinity)
  • combine all these in hierarchy
  • loadbalance to everywhere but some zones (shouldn't apply to the zones excluded)

Ideally it would be nice to not have to define multiple "mirrored" policies to define symmetry

lahabana avatar Oct 02 '23 17:10 lahabana

  • We should be careful at interacting correctly with MeshHTTPRoute when using subsets (xref: https://github.com/kumahq/kuma/issues/3338)
  • Don't forget about zoneEgress
  • Maybe start by listing a bunch of use-cases and look at how to implement them in Envoy.

lahabana avatar Oct 03 '23 13:10 lahabana

1/ What about a service that has different tags in different zone/node? Example: Service_A (zone1) has a tag (k8s.io/node: a) Service_A (zone2) doesn’t have a tag k8s.io/node (it’s universal) And we have a policy:

targetRef:
  kind: Mesh
to:
  targetRef:
    kind: MeshService
    name: Service_A
  defaults:
    localityAware:
      - tag: k8s.io/node
        type: self
  1. How to behave when we have a list of the same tags is it OR or AND?
targetRef:
  kind: Mesh
to:
  targetRef:
    kind: MeshService
    name: Service_A
  defaults:
    localityAware:
      - tag: k8s.io/node
        type: self
      - tag: k8s.io/node
        type: affinity
        Group: [β€œa”, β€œb”]
  1. How should we behave when there are no tags?
  2. What to do when we don't match any rule?
    • there were no tags ( should we exclude this endpoint from the routing)
  3. Are the priorities in the same group the same? (We suspect that yes)
targetRef:
  kind: Mesh
to:
  targetRef:
    kind: MeshService
    name: Service_A
  defaults:
    localityAware:
      - tag: k8s.io/node
        type: self
      - tag: k8s.io/node
        type: affinity
        Group: [β€œa”, β€œb”]

Self is priority 0 Node a,b are priorities 1 or maybe all are priorities 0?

lukidzi avatar Oct 05 '23 07:10 lukidzi

  1. What about a service that has different tags in different zone/node? I think for loadBalancing inside a zone missing tags are just ignored
  2. How to behave when we have a list of the same tags is it OR or AND? I think it's just that the second entry has lower priority
  3. How should we behave when there are no tags? I think we just ignore the rule
  4. What to do when we don't match any rule? We should define the behaviour. I think it falls in the default locality aware LB behaviour which is enabled by default (at least for backward compatibility).
  5. Are the priorities in the same group the same? (We suspect that yes) Yes I think so

Come to think about it. It feels like we should have a strong difference between zone LocalityAware LB and multi-zone LB.

  • In zone localityAware you can use all labels except kuma.io/zone (as it doesn't make sense) and groups etc probably doesn't make sense.
  • In multi-zone LB you can only use kuma.io/zone and things like group makes sense.

Separating the 2 might also make the API clearer no?

lahabana avatar Oct 05 '23 11:10 lahabana

If we want to allow the use of any tag then it might limit them. I assume you can define some tags and define your subset by this. It seems like Mesh*Routes but on the cluster level.

lukidzi avatar Oct 05 '23 11:10 lukidzi

  1. What about a service that has different tags in different zone/node? I think for loadBalancing inside a zone missing tags are just ignored
  2. How to behave when we have a list of the same tags is it OR or AND? I think it's just that the second entry has lower priority
  3. How should we behave when there are no tags? I think we just ignore the rule
  4. What to do when we don't match any rule? We should define the behaviour. I think it falls in the default locality aware LB behaviour which is enabled by default (at least for backward compatibility).
  5. Are the priorities in the same group the same? (We suspect that yes) Yes I think so

Come to think about it. It feels like we should have a strong difference between zone LocalityAware LB and multi-zone LB.

  • In zone localityAware you can use all labels except kuma.io/zone (as it doesn't make sense) and groups etc probably doesn't make sense.
  • In multi-zone LB you can only use kuma.io/zone and things like group makes sense.

Separating the 2 might also make the API clearer no?

It will probably make API simpler, but will it be easier to understand by user? Beside MeshHTTPRoute, you will have two different ways to define routing, maybe some of this behaviour could be implemented as part of MeshHTTPRoute?

I see basically two things here, one is simple locality aware routing, when you only care about sending traffic in a single physical location, with fallback to anywhere. The second is more of a routing based on tags with more sophisticated fallback (maybe this should be part of MeshHTTPRoute?)

Automaat avatar Oct 09 '23 07:10 Automaat

@Automaat could you show examples of these 2 options? To me MHR and this feature have no overlap as MHR create new clusters for each backendRef if it uses tags. If we implement this locality aware stuff it would work the same on these cluster subsets than it does on full clusters no?

lahabana avatar Oct 09 '23 09:10 lahabana

@lahabana what kind of examples would you like to see? From the API point of view or the use cases?

Maybe this does not fit exactly info MHR. But maybe we are trying to overcomplicate it? When I think about my past experiences from working on platform, I see the couple of use cases:

  1. route only to local datacenter with fallback to any
  2. route only to specific datacenter because some services are only deployed there (but this is how it works by default)
  3. route according to priority: az, region, geolocation like us/eu

This use cases can be simply done with API example we have right now:

Ad 1.

targetRef:
  kind: Mesh
to:
  targetRef:
    kind: MeshService
    name: Service_A
  defaults:
    localityAware:
      - tag: region
        type: self
        fallback: any

Ad 2.

targetRef:
  kind: Mesh
to:
  targetRef:
    kind: MeshService
    name: Service_A
  defaults:
    localityAware:
      - tag: region
        type: affinity
        groups: ["specifi-dc"]
        fallback: any

Ad 3.

targetRef:
  kind: Mesh
to:
  targetRef:
    kind: MeshService
    name: Service_A
  defaults:
    localityAware:
      - tag: availabilityZone
         type: self
      - tag: region
         type: self
      - tag: geo
         type: self

I am not so sure with third example, how to show priorities

In the meantime, I've made full circle in my mind and finished with the problem you stated that we could be dividing our infra in two ways, based on physical or logical boundaries. Maybe we can assume that we are always working on logical boundaries and keep it as single config? Without dividing it into: zone LocalityAware LB and multi-zone LB?

On the other hand, we are simply defining routing rules based on tags, so locality part of it exists only in the head of person that is creating this configuration. It could exist without locality in name and would work the same

Automaat avatar Oct 09 '23 10:10 Automaat