💡 Allow configuration of a source IP pool to ensure there are enough ports to proxy large numbers of requests

[zintrepid]

Describe the feature you'd like We would like to be able to configure cloudflared to use a pool of IPs when communicating with the origin server to avoid exhausting all ports between Established and Time-Wait connections.

We receive a large number of requests per second across two load balancers. Further, we can receive 10s of thousands of requests in a single second during a DDoS event. Before moving to the tunnel, incoming requests to the load balancers originated from a fairly large number of source IPs in CloudFlare's IP blocks (https://www.cloudflare.com/ips/). After moving to the tunnel, we only have one source IP per cloudflared node.

Describe alternatives you've considered We have currently programmed our load-balancers to have 10 IP addresses each, instead of one, to ensure a large space of IP/port pairs. This has increased the complexity of our load balancer configuration, and caused additional complexity in our IP configuration. It would be simpler and more maintainable for us to use a pool of IPs at the cloudflare tunnel node, and return our load balancer's to their original configuration. Further, we would not have to reconfigure our load balancer's other virtual hosts as well, which we have not yet done.

[DigitalCyberSoft]

Assuming you're not on Windows... Cloudflared connects out with QUIC, you can increase the number of connections with ha-connections

On the internal origin side, if cloudflared is running on the same machine you could use a unix domain sockets. That gets you 2^20 connections per socket.

If that isn't an option, or you're on Windows, then issue #352 addresses this

[iBug]

@DigitalCyberSoft I may disagree with your last paragraph. #352 focuses on connections towards Cloudflare Edge, not upstream connections towards origin servers.

[iBug]

I believe your goal can be achieved using Linux multipath routing. It seems like a better idea than adding cloudflared support for this: The code base is too complex for this feature, or you can come up with a PR.

With kernel >= 4.14, you can add 10 IPs on the CF tunnel node, then follow this answer on Server Fault to set up multipath routing, with each nexthop having a separate source IP (e.g. nexthop via 192.0.0.1 src 192.0.2.1, ... src 192.0.2.2 to ... src 192.0.2.10). You need to set net.ipv4.fib_multipath_hash_policy = 1. You may want to leverage the nexthop objects (ip-nexthop(8)) and use nhid for routing, which can bring you easier configuration management.

With this setup, your load balancers only need 1 IP per node, and only the CF tunnel node requires supplementary addresses.