kuma
kuma copied to clipboard
Proper support for locality weighted load balancing
Summary
Today locality aware load balancing is implemented using priorities - local=0, remote=1. We couldn't add Locality weighted load balancing because it's not compatible with lb subsetting that was used for TrafficRoutes. Since we don't use subsetting anymore (every TrafficRoute.Split is a separate cluster) probably we can properly support locality weighted load balancing.
This issue aims to track the progress of investigation around the possibility to support locality weighted load balancing.
ππ» π
This issue was inactive for 30 days it will be reviewed in the next triage meeting and might be closed. If you think this issue is still relevant please comment on it promptly or attend the next triage meeting.
This needs to be implemented with multiple layers in mind.
For example including k8s: https://kubernetes.io/docs/concepts/services-networking/topology-aware-hints/
Ideally if we have topology.kubernetes.io/zone
and multiple Kuma zones it should be possible to have 3 priorities:
- same k8s zone and same Kuma zone
- different k8s zone but same Kuma zone
- different Kuma zone
The traffic would progressively migrate from the closest network to the furthest as the service degrades.
This should be potentially configurable on a per route basis as a list of tags .e.g:
type: TrafficRoute
name: route-all-default
mesh: default
sources:
- match:
kuma.io/service: '*'
destinations:
- match:
kuma.io/service: '*'
conf:
localityAwareLoadBalancingPriorities: [topology.kubernetes.io/zone, kuma.io/zone] # If this is unset traffic will remain stuck on the local zone?
xref #3456 which is probably going to be implemented at the same time.
Triage: This might also mean that standalone can have localityAwareLoadBalancing too.
Note: This should also include exposing the overprovisioning_factor
value
This issue was inactive for 30 days it will be reviewed in the next triage meeting and might be closed. If you think this issue is still relevant please comment on it promptly or attend the next triage meeting.
This issue was inactive for 30 days it will be reviewed in the next triage meeting and might be closed. If you think this issue is still relevant please comment on it promptly or attend the next triage meeting.
This issue was inactive for 30 days it will be reviewed in the next triage meeting and might be closed. If you think this issue is still relevant please comment on it promptly or attend the next triage meeting.
This issue was inactive for 30 days it will be reviewed in the next triage meeting and might be closed. If you think this issue is still relevant please comment on it promptly or attend the next triage meeting.
This issue was inactive for 30 days it will be reviewed in the next triage meeting and might be closed. If you think this issue is still relevant please comment on it promptly or attend the next triage meeting.
This issue was inactive for 90 days. It will be reviewed in the next triage meeting and might be closed. If you think this issue is still relevant, please comment on it or attend the next triage meeting.
This issue was inactive for 90 days. It will be reviewed in the next triage meeting and might be closed. If you think this issue is still relevant, please comment on it or attend the next triage meeting.
This issue was inactive for 90 days. It will be reviewed in the next triage meeting and might be closed. If you think this issue is still relevant, please comment on it or attend the next triage meeting.
Potentially related upstream k8s work:
- https://github.com/kubernetes/kubernetes/pull/116522
- https://github.com/kubernetes/enhancements/issues/2433
This issue was inactive for 90 days. It will be reviewed in the next triage meeting and might be closed. If you think this issue is still relevant, please comment on it or attend the next triage meeting.
In a quick brainstorm with @johnharris85 we came up with something like:
targetRef:
kind: Mesh
to:
targetRef:
kind: MeshService
name: foo
defaults:
localityAware:
- tag: k8s/node
type: 'self'
- tag: k8s/az
type: 'self'
- tag: kuma.io/zone
type: 'affinity'
groups:
- ['eu-east', 'eu-west']
- ['us-east', 'us-west']
fallback: 'any'
- tag: kuma.io/zone
type: 'affinity'
groups:
- ['eu-east', 'eu-west', 'eu-central']
- ['us-east', 'us-west', 'us-central']
fallback: 'fail'
We really need to first enumerate all the use-cases we want to deal with.
Some useful envoy stuff:
https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/upstream/aggregate_cluster https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/upstream/load_balancing/priority#arch-overview-load-balancing-priority-levels
Use cases:
- loadbalance within a zone to favour traffic on the same node, on the same AZ, on the same rack (for universal)
- disable disabling all cross zone traffic for a service
- only failover to one zone from another (grouped affinity)
- combine all these in hierarchy
- loadbalance to everywhere but some zones (shouldn't apply to the zones excluded)
Ideally it would be nice to not have to define multiple "mirrored" policies to define symmetry
- We should be careful at interacting correctly with MeshHTTPRoute when using subsets (xref: https://github.com/kumahq/kuma/issues/3338)
- Don't forget about zoneEgress
- Maybe start by listing a bunch of use-cases and look at how to implement them in Envoy.
1/ What about a service that has different tags in different zone/node?
Example:
Service_A (zone1) has a tag (k8s.io/node: a
)
Service_A (zone2) doesnβt have a tag k8s.io/node
(itβs universal)
And we have a policy:
targetRef:
kind: Mesh
to:
targetRef:
kind: MeshService
name: Service_A
defaults:
localityAware:
- tag: k8s.io/node
type: self
- How to behave when we have a list of the same tags is it OR or AND?
targetRef:
kind: Mesh
to:
targetRef:
kind: MeshService
name: Service_A
defaults:
localityAware:
- tag: k8s.io/node
type: self
- tag: k8s.io/node
type: affinity
Group: [βaβ, βbβ]
- How should we behave when there are no tags?
- What to do when we don't match any rule?
- there were no tags ( should we exclude this endpoint from the routing)
- Are the priorities in the same group the same? (We suspect that yes)
targetRef:
kind: Mesh
to:
targetRef:
kind: MeshService
name: Service_A
defaults:
localityAware:
- tag: k8s.io/node
type: self
- tag: k8s.io/node
type: affinity
Group: [βaβ, βbβ]
Self is priority 0 Node a,b are priorities 1 or maybe all are priorities 0?
- What about a service that has different tags in different zone/node? I think for loadBalancing inside a zone missing tags are just ignored
- How to behave when we have a list of the same tags is it OR or AND? I think it's just that the second entry has lower priority
- How should we behave when there are no tags? I think we just ignore the rule
- What to do when we don't match any rule? We should define the behaviour. I think it falls in the default locality aware LB behaviour which is enabled by default (at least for backward compatibility).
- Are the priorities in the same group the same? (We suspect that yes) Yes I think so
Come to think about it. It feels like we should have a strong difference between zone LocalityAware LB and multi-zone LB.
- In zone localityAware you can use all labels except
kuma.io/zone
(as it doesn't make sense) and groups etc probably doesn't make sense. - In multi-zone LB you can only use
kuma.io/zone
and things like group makes sense.
Separating the 2 might also make the API clearer no?
If we want to allow the use of any tag then it might limit them. I assume you can define some tags and define your subset by this. It seems like Mesh*Routes but on the cluster level.
- What about a service that has different tags in different zone/node? I think for loadBalancing inside a zone missing tags are just ignored
- How to behave when we have a list of the same tags is it OR or AND? I think it's just that the second entry has lower priority
- How should we behave when there are no tags? I think we just ignore the rule
- What to do when we don't match any rule? We should define the behaviour. I think it falls in the default locality aware LB behaviour which is enabled by default (at least for backward compatibility).
- Are the priorities in the same group the same? (We suspect that yes) Yes I think so
Come to think about it. It feels like we should have a strong difference between zone LocalityAware LB and multi-zone LB.
- In zone localityAware you can use all labels except
kuma.io/zone
(as it doesn't make sense) and groups etc probably doesn't make sense.- In multi-zone LB you can only use
kuma.io/zone
and things like group makes sense.Separating the 2 might also make the API clearer no?
It will probably make API simpler, but will it be easier to understand by user? Beside MeshHTTPRoute, you will have two different ways to define routing, maybe some of this behaviour could be implemented as part of MeshHTTPRoute?
I see basically two things here, one is simple locality aware routing, when you only care about sending traffic in a single physical location, with fallback to anywhere. The second is more of a routing based on tags with more sophisticated fallback (maybe this should be part of MeshHTTPRoute?)
@Automaat could you show examples of these 2 options? To me MHR and this feature have no overlap as MHR create new clusters for each backendRef if it uses tags. If we implement this locality aware stuff it would work the same on these cluster subsets than it does on full clusters no?
@lahabana what kind of examples would you like to see? From the API point of view or the use cases?
Maybe this does not fit exactly info MHR. But maybe we are trying to overcomplicate it? When I think about my past experiences from working on platform, I see the couple of use cases:
- route only to local datacenter with fallback to any
- route only to specific datacenter because some services are only deployed there (but this is how it works by default)
- route according to priority: az, region, geolocation like us/eu
This use cases can be simply done with API example we have right now:
Ad 1.
targetRef:
kind: Mesh
to:
targetRef:
kind: MeshService
name: Service_A
defaults:
localityAware:
- tag: region
type: self
fallback: any
Ad 2.
targetRef:
kind: Mesh
to:
targetRef:
kind: MeshService
name: Service_A
defaults:
localityAware:
- tag: region
type: affinity
groups: ["specifi-dc"]
fallback: any
Ad 3.
targetRef:
kind: Mesh
to:
targetRef:
kind: MeshService
name: Service_A
defaults:
localityAware:
- tag: availabilityZone
type: self
- tag: region
type: self
- tag: geo
type: self
I am not so sure with third example, how to show priorities
In the meantime, I've made full circle in my mind and finished with the problem you stated that we could be dividing our infra in two ways, based on physical or logical boundaries. Maybe we can assume that we are always working on logical boundaries and keep it as single config? Without dividing it into: zone LocalityAware LB and multi-zone LB
?
On the other hand, we are simply defining routing rules based on tags, so locality part of it exists only in the head of person that is creating this configuration. It could exist without locality in name and would work the same