gateway-api

gateway-api copied to clipboard

What would you like to be added: An option for defining a list of IP ranges that should be allowed/denied to call a certain HTTPRoute

Why this is needed: When using the Gateway API for http requests it is possible to create a single http handler for all http traffic, which is delegated to the various backends. In some instances it might be necessary to add additional ACL's in order to control which source IP's are allowed to call those endpoints. I might have a route for https://gw.company.tld/public that should be accesible for all but https://gw.company.tld/internal that I want to restrict to IP ranges owned by my employer.

In ingress controllers that can typically be configured by adding an annotiation on the Ingress object eg: nginx.ingress.kubernetes.io/whitelist-source-range: 10.0.0.0/24,172.10.0.1

There are several problems with this approach that I think can be solved better in Gateway API

• the annotation name is not standardized - so the whitelist is not portable accross ingress implementations
• the behaviour is not standardized - some implementations only have an allow/whitelist whereas others have both an allowlist and a denylist
• enummerating multiple IPs or ranges in a single annotation becomes harder to read as the list grows

Ref: https://haproxy-ingress.github.io/docs/configuration/keys/#allowlist https://www.haproxy.com/documentation/kubernetes/latest/configuration/ingress/#whitelist https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#whitelist-source-range

Thanks for raising this issue @hoerup! When discussing this in relation to GEP #735, one of the questions we had was where this configuration should belong (more in #1127). Is it important that this is tied to a route? If so, how would merging and conflict resolution work? If not, would it be sufficient to tie this to a Gateway? Should it apply to the full Gateway or per listener?

As I see it, the IP ACL should definitely be tied to a route - in order to give the operator the wides flexibility so (s)he kan define multiple HTTPRoutes each with it's own ACL's although an option to apply it on gateway/listener as well might me usefull in other scenarios?

While looking at the current (0.4.2) spec and wondering where to put this im thinking that it should probably go into a new acls field in HTTPRouteRule. That way an incomming http request will be matched first against the hostnames in HTTPRouteSpec and any matches defined in HTTPRouteRule.matches. When a route has been selected then we can apply any ACL's to determine whether the client is allowed to access this route and if not reject it with a HTTP 403

Another design issue is whether the IP ACL should be inline sub-object OR perhaps be an new top-level Kind. If it is a new Kind then an operator could define a set of IP ACL's at a central place and refer to them from the various route's as needed

Filters on HTTPRoute were meant to solve exactly this class of problem and I think we should explore that approach a bit more before additions to the core API.

A feature requests to specific implementation to add a custom filter to support this use case makes a lot of sense as the first step. If multiple implementations add support for such a capability, then we can discuss moving the feature into an "Extended" conformance level. And then eventually "Core" if the time comes for it.

Would that help your use-case @hoerup?

hbagdi: Yes - i think that when I originally wrote this i hadn't completely understood the purpose of filters. But yes a HttpACLFilter would be great

I wonder if the suggested filter approach would be manageable. If it is only necessary to apply ACLs to one rule, then it looks simple enough. However, if it is necessary to apply ALCs to more than one rule, then there will be duplicated ACLs spread across one resource. Or even multiple resources. Maintaining those duplicated ACLs will be a burden imho.

Additionally, should ACLs be the responsibility of the developer persona (who owns HTTPRoute) or the cluster operator?

I think there are two classes of IP ACLs, per-app and per-infrastructure.

Using HTTPRoute Filters to handle per-app sounds like a good fit, but won't scale well if you want to apply the same ACLs across lots of things.

To be honest, per-infrastructure IP ACLs are probably better handled either by something in the Gateway spec, or by something that attaches to the Gateway (like a Policy resource). For simplicity, a Policy resource could mandate the use of particular Filters for every route attached to a Gateway. But this design currently only exists in my head. (I have a TODO to update the Policy documentation with some more examples to illustrate this sort of use-case).

It feels like there may be some substantial overlap here with NetworkPolicy ingress rules (if it could be explicitly applied to a Gateway resource instead of a pod selector?) for the whole-infra case.

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

/remove-lifecycle stale

We need to triage and evaluate this further to determine if this is something that we want to do, but to that end it will be low priority until v1.0.0/GA is complete and don't believe we will have bandwidth for it until then.

I recently worked on a similar feature in contour, and thought we had some good discussions that could be relevant for the Gateway APIs.

Some of the highlights:

• Filtering can be configured per virtualhost (Gateway) but overridden per route (HttpRoute / GrpcRoute)
• An individual ip filter is either an allowlist or denylist but not both
• There was a need to support extra configuration to indicate how the incoming ip address should be fetched (in envoy this is distinguished by direct_remote_ip and remote_ip, and the number of hops to trust if using X-Forwarded-For).

Design: https://github.com/projectcontour/contour/blob/2c6015d30004508661ee2cf086354d91bc6d0986/design/ip-filtering-design.md Docs: https://projectcontour.io/docs/1.25/config/ip-filtering/ API Bikeshed: https://github.com/projectcontour/contour/pull/4990#issuecomment-1408693088

Using this in Contour HTTPProxy. Would be nice to have this in the Gateway API. Not being able to limit outside access to a specific Gateway is painful.

Just a note that this has basically gone stale, if someone is interested in championing it forward and working on a proposal please let us know.

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle rotten
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

/remove-lifecycle rotten

/cc @tssurya @npinaeva

You may be interested in this re: Network Policy