Expose routing information for VO invocations for L7 load balancer

[tillrohrmann]

Users have asked to have some control over routing VO calls to a specific service endpoint. Such functionality could allow to cache information on this service endpoint for future calls. One idea could be to expose the key (probably a hash of it) as a header which could be used by a L7 load balancer to make routing decisions. This would only work in this specific setup, though. A bit of an open question is how service calls would be handled. Maybe they wouldn't have this header set.

[slinkydeveloper]

Some research:

• CloudFlare^1 and Envoy^2 lets you set a custom header you want to use for session affinity. They also provide another feature, that lets you reuse the endpoint the load balanced picked the first time, based on the cookie header.
• NGINX: I could find only the feature that lets the load balancer set the cookie header?!
• HAProxy has huge amount of flexibility, with this concept of stick tables^3. The "sticky string" can be of any size, but they default it to 32 chars long

Based on these considerations, I would say:

• Let's introduce an option to enable this header and pick the name, let's not enable it by default though.
• Let's pick some hashing for service name/service key, with the following properties:
  • Optimized for speed
  • String size perhaps we need just 64 chars?
  • Collisions are fine

[peterbourgon]

Anything that would be looking at a request header to do e.g. session affinity would necessarily need to hash that header value anyways, so, maybe dumb question, but why not use the service name/key directly?

[slinkydeveloper]

Anything that would be looking at a request header to do e.g. session affinity would necessarily need to hash that header value anyways, so, maybe dumb question, but why not use the service name/key directly?

Isn't it better to use a hash to send less stuff over the network (and with a more predictable size)? Seems things like haproxy (looking at their docs of sticky tables) will truncate before hashing, so a sufficiently long service name might be a problem there.

[peterbourgon]

HTTP header values are larger than the stick table key limits, so anyone who would use a header as part of a key would necessarily be applying a hash on their side anyway, e.g.

http-request track-sc0 req.hdr(X-User-ID),hash

And same would be true for all of the other systems, I believe.

[slinkydeveloper]

Caddy allows to configure the header we want: https://caddyserver.com/docs/caddyfile/directives/reverse_proxy#load-balancing