caddy-ratelimit
caddy-ratelimit copied to clipboard
mholt
Dynamic zone key for network block of {http.request.remote.host} with certain prefix
When used as the key of a dynamic zone, can {http.request.remote.host} be reduced to its network block for a certain prefix?
Assuming http.request.remote.host is 1.2.3.4, and a function is used reducing it to /24, requests (and rate-limiting) to any address in the range 1.2.3.0-255 would be grouped.
"key": "reduce_to_network_block({http.request.remote.host}, '/24')",
What if there was a placeholder:
{http.request.remote.host/N} where N could be any bit length that mapped the IP (if an IP, otherwise it would ignore the /N) to, say, 1.2.3.0 for anything in the 1.2.3.0-255 range?
I can implement that upstream in Caddy if that's sufficient for you.
What if it's IPv6 though? Need ranges for both cases I think. Kinda tricky.
There must be a way we can set up a var with the right value.
@francislavoie I don't think the https://pkg.go.dev/net/netip package cares. It's just more bits, right?
My point is, you might want to configure 16 bits for IPv4 but 32 bits for IPv6, like we have in the log ip_mask filter https://caddyserver.com/docs/caddyfile/directives/log#ip-mask
True; maybe we could support a /N,M syntax or something; let's wait for a feature request.
Oh what the hey. I did it anyway: https://github.com/caddyserver/caddy/commit/9873ff9918224f56604048ea0ed3d3ae3953939e
@kellytk Here's what you want: {http.request.remote.host/24,32} (for example; the ,32 is optional if you want both bit lengths for IPv4 and IPv6)
@mholt This is very exciting; especially the IPv6 support!
I wish to employ this feature in a layered fashion, ala contrived and simplified example: (IPv6 omitted as I don't yet have experience with it)
- {http.request.remote.host/24} window:1s max_events:2,560
- {http.request.remote.host/16} window:1s max_events:65,536
- {http.request.remote.host/8} window:10s max_events:16,777,216
One request from 1.2.3.4 would:
- Apply a +1 event to the 1.2.3.0-255 zone key bucket.
- Another +1 event to the 1.2.0-255.0-255 bucket.
- And finally another +1 event to the 1.0-255.0-255.0-255 bucket.
Is that how this feature, as implemented, is designed to function?
This would enable me to allow burst traffic from individual IPs and small blocks, such as CGNAT, while still applying the events to the quotas of the larger containing blocks. The rationale is that individual IPs and small blocks can legitimately be hot, however amortizing events across larger blocks should yield more moderate figures. If large blocks are also hot, it's possibly an attack scenario.
To thwart my thinking an attacker would need to utilize more evenly distributed addresses from the global space. That can be 'easily' mitigated with active monitoring and dynamic updating of these rate limit policies.
@kellytk Yes, I believe that should work. The placeholder {http.request.remote.host/24} for example would reduce 192.168.55.123 to a key of 192.168.55.0/24 -- /16 would become 192.168.0.0/16 and /8 would output 192.0.0.0/8.
You could hard-code those zone keys to test, if you know the IP range you'll be testing with. Then use the placeholder and observe the same behavior.
@mholt Excellent, thank you! Will tests be added to verify the behavior and ensure there're no regressions in the future?
@kellytk I did write unit tests: https://github.com/caddyserver/caddy/commit/9873ff9918224f56604048ea0ed3d3ae3953939e#diff-f7d3f5ac378699eb31c965343929663ac9504606500b9b4f81f10d371bfc9677
We can always add more later if needed.
What are you using this for? Just curious.
@mholt Thank you! I'm using Caddy as the user-facing reverse proxy to Rust-based 'apps' I write.
With enhancements such as this and https://github.com/caddyserver/caddy/issues/4558 Caddy is proving pleasantly mature. Thank you once again.