Dynamic load balancing

[murali-reddy]

IPVS has this nice LB method http://kb.linuxvirtualserver.org/wiki/Dynamic_Feedback_Load_Balancing_Scheduling

Which is all the more relevent in case of distrubuted load balancing requirements of 'ClusterIP' and NodePort service types. Each node doing load balancing in round robin fashion has below limitations.

• if connections are mixture of short lived and long lived then we will end up in imbalance
• each node is acting on the local connection information it has. Due to distrubuted nature each node ends up doing load balancing without taking account of other nodes or global state

Here is the proposal:

• Implement monitor daemon that collects connection per endpoint from the node using contrack
• use in-memory distrubuted key-value store to persist the data.
• use the global view of connections to dynamically adjust the weights on each node

[bzub]

Initial Implementation

The monitor daemon could weight real-servers with:

• Standard pod metrics provided by Heapster
• Readiness probe response times (if I can find that information exposed within kubernetes)

These kubernetes specific sources should alleviate the need for kube-router to maintain its own distributed data store to share weight variables. For distributed configuration between kube-router instances a ConfigMap could be updated by a daemon/Pod responsible for calculating load-balancing weights, and kube-router could watch/consume that config.

Research and Justification

There are several ways to get at least pod system-level load metrics in Kubernetes. Application-level load metrics are also increasingly common in Kubernetes apps. Especially true since it's a requirement to expose these metrics to use Horizontal Pod Autoscaling

So here's some potential sources of load metrics that our monitor daemon could support:

• Prometheus scrape endpoints
• Pod and Container Readiness Probe events (response times)
• cAdvisor builtin and custom metrics
• Ingress Controller metrics (nginx, haproxy, traefik). In real use cases these could be independent or fed through Heapster/cAdvisor or Prometheus

Heapster is a relatively mature and popular cluster addon that collects from several of the above mentioned sources, so it could be a definitive source for system-level load metrics. There is also an effort to standardize this type of information in the apiserver with a metric endpoint. This API might be more apt for app-level load metrics. More info on that in this issue, and implementation work in this repo.

[SEJeff]

There is an ingress doing something vaguely similar, but they are storing traffic stats in etcd.

[SEJeff]

Also, heapster has sort of been replaced by the metrics api. Additionally, groupcache might fit into your ideas of an in-memory k/v store with the benefits of scaling out

[aauren]

Closing as stale

[murali-reddy]

some design details on how cloudfare implemented dynamic load balancing

https://blog.cloudflare.com/unimog-cloudflares-edge-load-balancer/

current implementation in kube-router is static load balancing worth while to reconsider to truly load balncing

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[github-actions[bot]]

This issue was closed because it has been stale for 5 days with no activity.