Add a post translation hook for non-HTTP listener

[modatwork]

trafficstars

Description:

In the current extension design there is a PostHTTPListenerModify RPC for HTTP listener and a PostTranslateModify RPC for cluster and secret resources. We can not touch non-HTTP listeners until some policy is implemented in EG. Envoy has so many configuration that EG may not expose all the APIs. So adding non-HTTP listener to a post translation hook helps a lot.

One usage is to support transparent mode TCP proxy. Users could add Original Src Filter[^1] in the hook and handle the route related work in their own. Another example is to add a custom TLS handshaker[^2] to the listener.

[optional Relevant Links:]

[^1]: Original Src Filter: https://www.envoyproxy.io/docs/envoy/v1.29.0/api-v3/extensions/filters/listener/original_src/v3/original_src.proto#extensions-filters-listener-original-src-v3-originalsrc [^2]: custom_handshaker: https://www.envoyproxy.io/docs/envoy/v1.29.0/api-v3/extensions/transport_sockets/tls/v3/tls.proto.html

[arkodg]

this makes sense, definitions are in https://github.com/envoyproxy/gateway/blob/main/proto/extension/service.proto cc @AliceProxy @guydc who are using this feature today and can help guide this issue forward

[arkodg]

@modatwork one temporary workaround is to use EnvoyPatchPolicy https://gateway.envoyproxy.io/latest/user/envoy-patch-policy/

[shuji-2019]

But EnvoyPatchPolicy can't solve all problems.

For example, when MergeGateways is enabled, multi Gateway TLS Listeners are configured with same port and protocol but different hostname. Those Gateway Listeners will be translated into single xDS Listener, with each Gateway Listener as one Network Filter, distinguished by SNI.

If I want to configure each Network Filter with different custom handshaker by EnvoyPatchPolicy, that will be difficult. Because Network Filters are stored as array in xDS Listener, I can't tell which item in Network Filter array should be patched with specified custom handshaker.

xDS Listener below as example:

{
  "@type": "type.googleapis.com/envoy.config.listener.v3.Listener",
  "name": "default/gw-http/test",
  "address": {
    "socket_address": {
      "address": "0.0.0.0",
      "port_value": 8080
    }
  },
  "filter_chains": [
    {
      "filter_chain_match": {
        "server_names": [
          "test.example.com"
        ]
      },
      "filters": [
        {
          "name": "envoy.filters.network.http_connection_manager",
          "typed_config": {
            "@type": "type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager",
          }
        }
      ],
      "transport_socket": {
        "name": "envoy.transport_sockets.tls",
        "typed_config": {
          "@type": "type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext",
        }
      }
    },
    {
      "filters": [
        {
          "name": "envoy.filters.network.http_connection_manager",
          "typed_config": {
            "@type": "type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager",
          }
        }
      ],
      "transport_socket": {
        "name": "envoy.transport_sockets.tls",
        "typed_config": {
          "@type": "type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext",
        }
      }
    }
  ]
}

[liorokman]

But EnvoyPatchPolicy can't solve all problems.

+1

I suggest adding some more hook functions to the interface to support the additional listener types: TCP, and UDP listeners as well as HTTP.

[aoledk]

@arkodg After #3371 is done, can we add post translation hook to let extension server to modify TCP listener, also with policies in context.

[aoledk]

@arkodg When M HTTPS listeners and N TLS listeners co-exist on the same port, they will be translated to single xDS listener named X. Inside xDS listener, HTTPS listeners are translated to FilterChains set with HCM and TLS listeners are translated to FilterChains set with TCPProxy.

During translating, for each HTTPS listener, PostHTTPListenerModify will be invoked M times to modify the xDS listener X, along with selected extension server policies.

If we define a new hook, like PostTCPListenerModify, then it will be invoked N times after translating each TLS listener, to modify the xDS listener X, along with selected extension server policy.

These two hooks will have exact same function signature, and to modify the same xDS listener X.

So do we still need a new PostTCPListenerModify hook ? Or just reuse PostHTTPListenerModify for non-HTTP listener.

[arkodg]

yeah combining the 2 makes sense, wdyt @AliceProxy