💡Support for multiple replicas of a service under the same hostname

[christidis]

Describe the feature you'd like Is there support, or any intention to support, multiple replicas of a service under the same hostname in Zero Trust / cloudflared?

Assumming I have two replicas of a service running on instances http://vm01:8000 and http://vm02:8000. I would like to be able to expose them through foo.example.org with one (or more) cloudflared agents withoug having to also configure an LB software or proxy equivalent. Requests should be load balanced to the backends and healthchecks should be configurable. This is required for High Available applications exposed through Zero Trust.

Describe alternatives you've considered Maintain a proxy in front of the applications to handle the load balancing, backends healthchecks etc. Even for a software proxy (eg NGINX or Traefik) you still have to address the issue of multiple backeds for HA to avoid a SPOF. Not to mention the extra hop.

It would be great if one can avoid all that and have cloudflared support multiple replicas of the same service.

Additional context https://github.com/cloudflare/cloudflared/issues/417 can be considered as a related request (and a more complicated one). It has to do with exposing different applications running on different ports under the same hostname while this is for just supporting multiple replicas of the same service.

Also, in the Kubernetes world, this is not an issue because you can map Cloudflared services to Kubernetes service objects.

[sudarshan-reddy]

Requests should be load balanced to the backends

The replica model soft supports this already: https://developers.cloudflare.com/cloudflare-one/connections/connect-apps/install-and-setup/deploy-cloudflared-replicas/#about-cloudflared-replicas. A Load Balancer is still the recommended solution if you want more control over distribution of this traffic. But note that this is on a best-effort basis. That being said, this still ensures high availability because if any of your replicas goes down, traffic is distributed to the ones that remain.

[christidis]

Wait. The page you have linked is for HA to multiple cloudflared replicas. This is already supported and works as expected. If one cloudflared goes down it failovers to another cloudflared. Also, as you said, a CF Load Balancer can provide more control over traffic distribution etc. I fully agree on that.

But this issue is about supporting multiple replicas of any hosted service in cloudflared/Zero trust.

For example this, (currently not supported) config

ingress:
  - hostname: foo.example.org
    service:
      - http://vm01:8000
      - http://vm02:8000

How am I supposed to do this?

I have tried multiple cloudflared agents running, One in vm01 and One in vm02 each one pointing to the service running on the same VM, but when vm01:8000 for example goes down, traffic is still routed (through cloudflared running on vm01) to the backend resulting in 503 errors.

[sudarshan-reddy]

Thanks for clearing that up.

Let me try to understand your system better.

From your ingress example, I'm assuming these two services/replicas run behind the same tunnel? And thats why you use a proxy to balance traffic between them. Is my understanding of your system okay?

[christidis]

Right, they are in the same tunnel. And yes when using a proxy I load balance requests between them.

But even with proxies (on a HA setup with 2+ proxies) I carry over the same issue to the proxies.

There are solutions for making proxies HA with additional software like Pacemaker, Corosync, even keepalived & RR DNS but I was thinking it would be nice if cloudflared (which already supports multiple cloudflared in a HA setup) could handle multiple replicas for the tunnelled services as well.

[abelinkinbio]

As a follow-up, is your intent to remove the need for a LB solution altogether? I ask as the example you provided would be supported by pairing Tunnel + Cloudflare LB today. Instead of having a single tunnel, you could create two tunnels with two distinct ingress rules like so:

tunnel: tunnel_01
credentials-file: /root/.cloudflared/uuid01

ingress:
  - hostname: foo.example.org
    service:
      - http://vm01:8000

tunnel: tunnel_02
credentials-file: /root/.cloudflared/uuid02

ingress:
  - hostname: foo.example.org
    service:
      - http://vm02:8000

This would allow you to implement a failover configuration with Cloudflare LB where each Tunnel would be a unique origin.

[christidis]

Thanks @abelinkinbio for the suggestion. I will look into it more and most probably this is how I will do it. However, I still find this implementation meh..

What if instead of 2 replicas in my simplified example I had 10, or 20 or a dynamic number? Do I have to provision and manage 20 tunnels? And what if I need to scale the application up and down dynamically. Instead of a simple configuration change in cloudflared's config.yaml (taking advantage of the autoupdate option) I'll now have to manage multiple tunnel endpoints.

I beleive that having some kind of basic proxing capabilities within cloudflared (support a pool of replicas, healthchecks, headers etc) will simplify the whole setup. It will also allow to route everything though a single tunnel.

[abelinkinbio]

No problem at all. I'll continue to look into this and see if there is better automation to be implemented within the feature-set we do currently support. I've also noted this as a feature request internally.

[christidis]

This is to confirm that pairing each cloudflare Tunnel with Cloudflare LoadBalancer, and configuring a Pool of Tunnels as Origins with Healthcheck (Monitors) for their status worked well for my case, for HA & Failover. In addition, the whole configuration was fully supported in Terraform which is great!

I have also ended up using a Proxy (that I was planning to get rid off with tunnels) in front of my workloads to simplify the load balancer pool configuration and handle dynamic scaling cases on my backends.

Tunnels+LB is a good solution even though it may be a bit over-complicated solution for simple cases where you just need a single tunnel for exposing a couple of stateless applications with many replicas. Having some basic proxying support within cloudflared would be great.

I was about to close this with a Won't Fix resolution, but based on the latest comment this may be still a valid feature request, so I am leaving it open and up to the team to decide. Thank you everyone.

[michaelst]

Another use case for that this would help solve for my team is where we are running into the network limitations of outbound connections to the same ip, we are creating multiple k8s services to have more ips but we have to split the traffic to different domains so we can tell the cloudflare configuration to use the different ones. With this feature we could use a single domain to point at multiple ips to get around the network limitations.