Support Least Connections Load Balancing

[jamesrwu]

It would be awesome if Fabio supported least connections load balancing. There are cases when load balancing across multiple instances of the same service, some instances may become more overloaded than others due to a variety of factor such as heterogeneous hw, the nature of the request, etc. By using a least connections load balancing approach, we were able to see problems like this disappear. However, Fabio does not support it. I propose that Fabio read the load balancing algorithm via tags similar to the way the urlprefix tag is implemented. This allows flexibility in letting the application tell Fabio what algorithm to use.

[magiconair]

I've split this into two issues:

1. specify the balancing algorithm in the tag, e.g. urlprefix-/foo strategy=rr (see #251)
2. support least connections balancing (this ticket)

[magiconair]

Since I want to avoid keeping state across a cluster of fabio instances each fabio would have to keep its own LCR table per route. I'm trying to think whether this would defeat the purpose of this feature. I don't think so but I'm open for suggestions.

[jamesrwu]

+1 to not keeping state across a cluster of fabio instances. I don't think it is necessary and is more of a hard to implement "nice to have".

[magiconair]

I am wondering how least conn routing interacts with weighted routes. How should this work if you want lconn routing but send 5% of traffic to the services with the green tag?

I think to make this work you'd need a multi-stage selection process.

1. Find the services which serve the route
2. Find the set of services which should handle the request based on weight
3. Find the instance which should handle the request based on the strategy.

I could hack this into the existing code but this feels quite messy. Maybe a multi-stage filtering approach is better and more maintainable in the long run. A generic route filter function like the one below is probably more maintainable in the long run.

type RouteFilter = func(r []*Route) []*Route

hostFilter, prefixFilter, weightFilter, lconnFilter, rndFilter, rrFilter, ...

Start with all routes and apply the filters in a given order. At the end pick the first remaining target. This could create a lot of allocations but I might get around this.

I have to play with this a bit.

[jamesrwu]

I am not even sure if it makes sense to have least connections and weighted routes. They seem to be incompatible concepts by nature.

Although the approach above you suggested might work, I fail to imagine what use cases one might run into where they would need both least connections and weighted routes on the same service. From my POV, I would be ok with treating them as incompatible and exclusive settings.

[magiconair]

The use case is that you use weighted routing for blue/green deployments and the least conn target selection to handle heterogeneous setups. I think the two concepts are orthogonal.

[jamesrwu]

I see, I misunderstood weighted routing for load balancing weights. This does make sense...

[magiconair]

The LB weights are computed automatically.

[lucaslim]

Hey @magiconair, is least connection support on fabio still in the radar? :)

[djenriquez]

Hey @magiconair, is least connection support on fabio still in the radar? :)

Or @magiconair, possibly any other load balancing strategies for that matter? Latency-based load balancing would be nice.