Support FilterChain level Post xDS Hook

[aoledk]

Description:

Proposal to add FilterChain level Post xDS Hook.

WHY?

Giving three Gateway Listeners, and suppose Envoy has supported com.eg.filter_test HTTP filter and handshaker_test custom handshaker.

listeners:
  - name: http
    protocol: HTTP
    port: 80
  - name: https-foo
    protocol: HTTPS
    hostname: foo.eg.com
    port: 443
    tls: ...
  - name: https-bar
    protocol: HTTPS
    hostname: bar.eg.com
    port: 443
    tls: ...

If I have these goals:

1. Enable com.eg.filter_test HTTP filter on HCM translated by http Listener.
2. Enable handshaker_test custom handshaker on FilterChain translated by https-foo Listener.
3. Enable com.eg.filter_test HTTP filter on HCM translated by https-bar Listener.

After all of xDS translations including xDS Hook, xDS resources should archive my goals.

http Listener will be translated to xDS Listener similar like this:

default_filter_chain:
  filters:
    - http_connection_manager:
        http_filters:

          # Add this http filter via xDS Hook API
          - com.eg.filter_test

          - envoy.filters.http.router

https-foo and https-bar Listeners will be translated to xDS Listener similar like this:

filter_chains:
  - filters:
      - http_connection_manager:
          http_filters:
            - envoy.filters.http.router
    transport_socket:
      downstream_tls_context:
        common_tls_context:

          # Add this handshaker via xDS Hook API
          custom_handshaker: handshaker_test

    filter_chain_match:
      server_names:
        - foo.eg.com

  - filters:
      - http_connection_manager:
          http_filters:

            ## Add this http filter via xDS Hook API
            - com.eg.filter_test

            - envoy.filters.http.router
    transport_socket:
      downstream_tls_context:
        common_tls_context
    filter_chain_match:
      server_names:
        - bar.eg.com        

In EG, HTTP Listener is translated to default_filter_chain of xDS Listener, HTTPS Listeners are translated to distinct FilterChain in filter_chains of xDS Listener. Each Gateway Listener corresponds to single FilterChain in xDS Listener.

So if I want to goals above, context indicating which FilterChain will be modified in the xDS Listener should be passed to extension server implements HTTPListener Hook.

Another way is to support FilterChain Hook, after FilterChain is translated from Gateway Listener, EG finds Policies attaching to this Gateway Listener and pass it to extension server.

[optional Relevant Links:]

Any extra documentation required to understand the issue.

[liorokman]

So what you're looking for is to update the EnvoyGatewayExtension interface and add an additional method?

Something like:

service EnvoyGatewayExtension {
    rpc PostRouteModify (PostRouteModifyRequest) returns (PostRouteModifyResponse) {};
    rpc PostVirtualHostModify(PostVirtualHostModifyRequest) returns (PostVirtualHostModifyResponse) {};
    rpc PostHTTPListenerModify(PostHTTPListenerModifyRequest) returns (PostHTTPListenerModifyResponse) {};
    rpc PostTranslateModify(PostTranslateModifyRequest) returns (PostTranslateModifyResponse) {};

    // PostFilterChainModify provides a way for extensions to modify a filter chains generated by Envoy Gateway 
    // before it is finalized. 
    rpc PostFilterChainModify(PostFilterChainModifyRequest) returns (PostFilterChainModifyResponse) {};
}

The semantics could be that before PostHTTPListenerModify is called, then PostFilterChainModify is called for each of the available filter chains including the default filter chain.

[aoledk]

So what you're looking for is to update the EnvoyGatewayExtension interface and add an additional method?

Yes, additional method is used to control filter chain translated from Gateway listener, because xDS listener may have multiple filter chains.

The semantics could be that before PostHTTPListenerModify is called, then PostFilterChainModify is called for each of the available filter chains including the default filter chain.

It'll be better if PostHTTPListenerModify is called just after xDS listener is created. And when each filter chain is created from Gateway listener, PostFilterChainModify is called, then insert into xDS listener.

Because if N Gateway listeners will be translated into single xDS listener, there will be N filter chains in the final xDS listener, inserted one by one. Until translation is done, PostHTTPListenerModify will be called N times, the i (1 <= i <= N ) round call carry i filter chains. It has additional data transfer overhead if N is bigger.

[aoledk]

Or we can set name for every FilterChain, irListenerName(listener) ^1 as name, used by PostHTTPListenerModify hook to tell which FilterChain in HTTP Listener should be modified.

[aoledk]

Since #3522 has implemented as each xDS listener will be modified only once via hook, along with all polices related to Gateway listeners translated to this xDS listener. FilterChain level hook will not be needed anymore.

[modatwork]

It's hard even impossible for an extension server to find the original target of a policy when xDS listener hook is called. Here is an example.

• mergeGateways is enabled
• 2 gateways with different labels share the same gateway class, each gateway has an HTTPS listener on port 8443
• a policy using a label selector which only select one of the 2 gateways

The 2 gateways will be translated to 2 filter chains on the same xDS listener. Then extension server won't be able to tell which filter chain should be patched.