gateway-api icon indicating copy to clipboard operation
gateway-api copied to clipboard

conformance: `TLSRoute` with Terminate mode

Open phuhung273 opened this issue 3 months ago • 28 comments

What type of PR is this? /kind test /area conformance-test

What this PR does / why we need it: This PR introduces basic same namespace conformance tests for TLSRoute with Terminate mode

Contour test
$ go test -v ./conformance -run TestConformance/TLSRouteTerminateSimpleSameNamespace -args         --gateway-class=contour         --supported-features=Gateway,TLSRoute,TLSRouteModeTerminate
=== RUN   TestConformance
    conformance.go:129: Running conformance tests with:
    conformance.go:147:   GatewayClass: contour
    conformance.go:148:   Cleanup Resources: true
    conformance.go:149:   Debug: false
    conformance.go:150:   Enable All Features: false
    conformance.go:151:   Supported Features: [Gateway TLSRoute TLSRouteModeTerminate]
    conformance.go:152:   ExemptFeatures: []
    conformance.go:153:   ConformanceProfiles: []
    suite.go:364: 2025-10-20T16:19:17.423656894Z: Test Setup: Ensuring GatewayClass has been accepted
    suite.go:370: 2025-10-20T16:19:17.426715459Z: Test Setup: Applying base manifests
    apply.go:275: 2025-10-20T16:19:17.431193734Z: Creating gateway-conformance-infra Namespace
    apply.go:275: 2025-10-20T16:19:17.437245559Z: Creating same-namespace Gateway
    apply.go:275: 2025-10-20T16:19:17.450504441Z: Creating same-namespace-with-https-listener Gateway
    apply.go:275: 2025-10-20T16:19:17.460037337Z: Creating all-namespaces Gateway
    apply.go:275: 2025-10-20T16:19:17.467488121Z: Creating backend-namespaces Gateway
    apply.go:275: 2025-10-20T16:19:17.476257877Z: Creating infra-backend-v1 Service
    apply.go:275: 2025-10-20T16:19:17.488303273Z: Creating infra-backend-v1 Deployment
    apply.go:275: 2025-10-20T16:19:17.498970525Z: Creating infra-backend-v2 Service
    apply.go:275: 2025-10-20T16:19:17.523116476Z: Creating infra-backend-v2 Deployment
    apply.go:275: 2025-10-20T16:19:17.542801376Z: Creating infra-backend-v3 Service
    apply.go:275: 2025-10-20T16:19:17.608387579Z: Creating infra-backend-v3 Deployment
    apply.go:275: 2025-10-20T16:19:17.66450806Z: Creating tls-backend Service
    apply.go:275: 2025-10-20T16:19:17.707566832Z: Creating tls-backend Deployment
    apply.go:275: 2025-10-20T16:19:17.721066884Z: Creating gateway-conformance-app-backend Namespace
    apply.go:275: 2025-10-20T16:19:17.737796568Z: Creating tls-backend Service
    apply.go:275: 2025-10-20T16:19:17.768281049Z: Creating tls-backend Deployment
    apply.go:275: 2025-10-20T16:19:17.783730522Z: Creating app-backend-v1 Service
    apply.go:275: 2025-10-20T16:19:17.80156872Z: Creating app-backend-v1 Deployment
    apply.go:275: 2025-10-20T16:19:17.823238397Z: Creating app-backend-v2 Service
    apply.go:275: 2025-10-20T16:19:17.851680654Z: Creating app-backend-v2 Deployment
    apply.go:275: 2025-10-20T16:19:17.86861977Z: Creating gateway-conformance-web-backend Namespace
    apply.go:275: 2025-10-20T16:19:17.879571982Z: Creating web-backend Service
    apply.go:275: 2025-10-20T16:19:17.917919134Z: Creating web-backend Deployment
    apply.go:275: 2025-10-20T16:19:17.932933836Z: Creating grpc-infra-backend-v1 Service
    apply.go:275: 2025-10-20T16:19:17.978316259Z: Creating grpc-infra-backend-v1 Deployment
    apply.go:275: 2025-10-20T16:19:17.996622706Z: Creating grpc-infra-backend-v2 Service
    apply.go:275: 2025-10-20T16:19:18.084959708Z: Creating grpc-infra-backend-v2 Deployment
    apply.go:275: 2025-10-20T16:19:18.119774583Z: Creating grpc-infra-backend-v3 Service
    apply.go:275: 2025-10-20T16:19:18.202215174Z: Creating grpc-infra-backend-v3 Deployment
    apply.go:275: 2025-10-20T16:19:18.223737001Z: Creating coredns Service
    apply.go:275: 2025-10-20T16:19:18.265497023Z: Creating coredns Deployment
    apply.go:275: 2025-10-20T16:19:18.289966878Z: Creating coredns ConfigMap
    suite.go:373: 2025-10-20T16:19:18.306292511Z: Test Setup: Applying programmatic resources
    apply.go:226: 2025-10-20T16:19:18.840637254Z: Creating certificate
    apply.go:226: 2025-10-20T16:19:19.552166057Z: Creating tls-validity-checks-certificate
    apply.go:226: 2025-10-20T16:19:19.871263947Z: Creating tls-passthrough-checks-certificate
    apply.go:226: 2025-10-20T16:19:24.885428Z: Creating tls-passthrough-checks-certificate
    apply.go:226: 2025-10-20T16:19:27.322204744Z: Creating tls-checks-ca-certificate
    apply.go:226: 2025-10-20T16:19:30.185282928Z: Creating tls-checks-certificate
    apply.go:226: 2025-10-20T16:19:30.416791676Z: Creating tls-terminate-checks-certificate
    apply.go:226: 2025-10-20T16:19:31.088395521Z: Creating mismatch-ca-certificate
    suite.go:395: 2025-10-20T16:19:31.102192909Z: Test Setup: Ensuring Gateways and Pods from base manifests are ready
    helpers.go:240: 2025-10-20T16:19:31.120754304Z: Ready condition set to False, expected True
    helpers.go:240: 2025-10-20T16:19:31.120942617Z: Ready was not in conditions list
    helpers.go:243: 2025-10-20T16:19:31.12096518Z: Pod gateway-conformance-infra/contour-backend-namespaces-5bff9757cd-dxqbk not ready yet
    helpers.go:240: 2025-10-20T16:19:32.115701699Z: Ready condition set to False, expected True
    helpers.go:240: 2025-10-20T16:19:32.115759354Z: Ready was not in conditions list
    helpers.go:243: 2025-10-20T16:19:32.115774544Z: Pod gateway-conformance-infra/contour-backend-namespaces-5bff9757cd-dxqbk not ready yet
    helpers.go:240: 2025-10-20T16:19:33.114752543Z: Ready condition set to False, expected True
    helpers.go:240: 2025-10-20T16:19:33.11483212Z: Ready was not in conditions list
    helpers.go:243: 2025-10-20T16:19:33.114847204Z: Pod gateway-conformance-infra/contour-backend-namespaces-5bff9757cd-dxqbk not ready yet
    helpers.go:240: 2025-10-20T16:19:34.115415962Z: Ready condition set to False, expected True
    helpers.go:240: 2025-10-20T16:19:34.11552668Z: Ready was not in conditions list
    helpers.go:243: 2025-10-20T16:19:34.115543211Z: Pod gateway-conformance-infra/contour-backend-namespaces-5bff9757cd-dxqbk not ready yet
    helpers.go:240: 2025-10-20T16:19:35.122058672Z: Ready condition set to False, expected True
    helpers.go:240: 2025-10-20T16:19:35.122162774Z: Ready was not in conditions list
    helpers.go:243: 2025-10-20T16:19:35.122262181Z: Pod gateway-conformance-infra/contour-backend-namespaces-5bff9757cd-dxqbk not ready yet
    helpers.go:240: 2025-10-20T16:19:36.119832596Z: Ready condition set to False, expected True
    helpers.go:240: 2025-10-20T16:19:36.11989534Z: Ready was not in conditions list
    helpers.go:243: 2025-10-20T16:19:36.119908361Z: Pod gateway-conformance-infra/contour-backend-namespaces-5bff9757cd-dxqbk not ready yet
    helpers.go:240: 2025-10-20T16:19:37.113961025Z: Ready condition set to False, expected True
    helpers.go:240: 2025-10-20T16:19:37.114018989Z: Ready was not in conditions list
    helpers.go:243: 2025-10-20T16:19:37.114030605Z: Pod gateway-conformance-infra/contour-backend-namespaces-5bff9757cd-dxqbk not ready yet
    helpers.go:240: 2025-10-20T16:19:38.112163633Z: Ready condition set to False, expected True
    helpers.go:240: 2025-10-20T16:19:38.112216047Z: Ready was not in conditions list
    helpers.go:243: 2025-10-20T16:19:38.112226898Z: Pod gateway-conformance-infra/contour-backend-namespaces-5bff9757cd-dxqbk not ready yet
    helpers.go:240: 2025-10-20T16:19:39.113153343Z: Ready condition set to False, expected True
    helpers.go:240: 2025-10-20T16:19:39.113199695Z: Ready was not in conditions list
    helpers.go:243: 2025-10-20T16:19:39.113209028Z: Pod gateway-conformance-infra/contour-backend-namespaces-5bff9757cd-dxqbk not ready yet
    helpers.go:240: 2025-10-20T16:19:40.121433255Z: Ready condition set to False, expected True
    helpers.go:240: 2025-10-20T16:19:40.121513978Z: Ready was not in conditions list
    helpers.go:243: 2025-10-20T16:19:40.121528779Z: Pod gateway-conformance-infra/contour-backend-namespaces-5bff9757cd-dxqbk not ready yet
    helpers.go:248: 2025-10-20T16:19:41.11871301Z: Gateways and Pods in gateway-conformance-infra, gateway-conformance-app-backend, gateway-conformance-web-backend namespaces ready
=== RUN   TestConformance/TLSRouteTerminateSimpleSameNamespace
    conformance.go:70: 2025-10-20T16:19:41.121024957Z: Applying tests/tlsroute-terminate-simple-same-namespace.yaml
    apply.go:275: 2025-10-20T16:19:41.126965079Z: Creating gateway-conformance-mqtt-test TLSRoute
    apply.go:275: 2025-10-20T16:19:41.135141171Z: Creating gateway-tlsroute-terminate Gateway
    apply.go:275: 2025-10-20T16:19:41.143450451Z: Creating mqtt-backend Service
    apply.go:275: 2025-10-20T16:19:41.167066401Z: Creating mqtt-backend Deployment
    apply.go:275: 2025-10-20T16:19:41.17618957Z: Creating mosquitto-config ConfigMap
    conformance.go:75: 2025-10-20T16:19:41.182865165Z: Running TLSRouteTerminateSimpleSameNamespace, relying on the following features: Gateway-standard, TLSRoute-experimental, TLSRouteModeTerminate-experimental
    helpers.go:216: 2025-10-20T16:19:41.207579744Z: Gateway gateway-conformance-infra/gateway-tlsroute-terminate expected observedGeneration to be updated to 1 for all conditions, only 0/2 were updated. stale conditions are: Accepted (generation 0), Programmed (generation 0)
    helpers.go:216: 2025-10-20T16:19:42.194811573Z: Gateway gateway-conformance-infra/gateway-tlsroute-terminate expected observedGeneration to be updated to 1 for all conditions, only 1/2 were updated. stale conditions are: Programmed (generation 0)
    helpers.go:240: 2025-10-20T16:19:43.193118602Z: Ready condition set to False, expected True
    helpers.go:240: 2025-10-20T16:19:43.193184839Z: Ready was not in conditions list
    helpers.go:243: 2025-10-20T16:19:43.193194408Z: Pod gateway-conformance-infra/envoy-gateway-tlsroute-terminate-gdsrp not ready yet
    helpers.go:240: 2025-10-20T16:19:44.194532529Z: Ready condition set to False, expected True
    helpers.go:240: 2025-10-20T16:19:44.194720998Z: Ready was not in conditions list
    helpers.go:243: 2025-10-20T16:19:44.194738361Z: Pod gateway-conformance-infra/envoy-gateway-tlsroute-terminate-gdsrp not ready yet
    helpers.go:240: 2025-10-20T16:19:45.194034568Z: Ready condition set to False, expected True
    helpers.go:240: 2025-10-20T16:19:45.194106464Z: Ready was not in conditions list
    helpers.go:243: 2025-10-20T16:19:45.19411659Z: Pod gateway-conformance-infra/envoy-gateway-tlsroute-terminate-gdsrp not ready yet
    helpers.go:240: 2025-10-20T16:19:46.193119647Z: Ready condition set to False, expected True
    helpers.go:240: 2025-10-20T16:19:46.193192332Z: Ready was not in conditions list
    helpers.go:243: 2025-10-20T16:19:46.193205526Z: Pod gateway-conformance-infra/envoy-gateway-tlsroute-terminate-gdsrp not ready yet
    helpers.go:240: 2025-10-20T16:19:47.195295278Z: Ready condition set to False, expected True
    helpers.go:240: 2025-10-20T16:19:47.195404789Z: Ready was not in conditions list
    helpers.go:243: 2025-10-20T16:19:47.19543438Z: Pod gateway-conformance-infra/envoy-gateway-tlsroute-terminate-gdsrp not ready yet
    helpers.go:240: 2025-10-20T16:19:48.19165631Z: Ready condition set to False, expected True
    helpers.go:240: 2025-10-20T16:19:48.191704771Z: Ready was not in conditions list
    helpers.go:243: 2025-10-20T16:19:48.191714849Z: Pod gateway-conformance-infra/envoy-gateway-tlsroute-terminate-gdsrp not ready yet
    helpers.go:240: 2025-10-20T16:19:49.192578265Z: Ready condition set to False, expected True
    helpers.go:240: 2025-10-20T16:19:49.192623684Z: Ready was not in conditions list
    helpers.go:243: 2025-10-20T16:19:49.192631992Z: Pod gateway-conformance-infra/envoy-gateway-tlsroute-terminate-gdsrp not ready yet
    helpers.go:240: 2025-10-20T16:19:50.192680116Z: Ready condition set to False, expected True
    helpers.go:240: 2025-10-20T16:19:50.192728681Z: Ready was not in conditions list
    helpers.go:243: 2025-10-20T16:19:50.192737405Z: Pod gateway-conformance-infra/envoy-gateway-tlsroute-terminate-gdsrp not ready yet
    helpers.go:240: 2025-10-20T16:19:51.191456859Z: Ready condition set to False, expected True
    helpers.go:240: 2025-10-20T16:19:51.191541488Z: Ready was not in conditions list
    helpers.go:243: 2025-10-20T16:19:51.191550263Z: Pod gateway-conformance-infra/envoy-gateway-tlsroute-terminate-gdsrp not ready yet
    helpers.go:248: 2025-10-20T16:19:52.192128262Z: Gateways and Pods in gateway-conformance-infra namespaces ready
    helpers.go:661: 2025-10-20T16:19:52.198582629Z: Conditions matched expectations
    helpers.go:661: 2025-10-20T16:19:52.198661039Z: Route gateway-conformance-infra/gateway-conformance-mqtt-test Parents matched expectations
=== RUN   TestConformance/TLSRouteTerminateSimpleSameNamespace/Simple_MQTT_TLS_request_matching_TLSRoute_should_reach_mqtt-backend
    tlsroute-terminate-simple-same-namespace.go:73: Establishing MQTT connection to host tls.example.com via 172.18.0.8:8883
    tlsroute-terminate-simple-same-namespace.go:94: Connected to MQTT broker
    tlsroute-terminate-simple-same-namespace.go:103: Subscribed, publishing test message...
    tlsroute-terminate-simple-same-namespace.go:97: Received message: Hello TLSRoute Terminate MQTT!
    tlsroute-terminate-simple-same-namespace.go:122: Round-trip test succeeded
=== NAME  TestConformance/TLSRouteTerminateSimpleSameNamespace
    apply.go:283: 2025-10-20T16:20:23.150697644Z: Deleting mosquitto-config ConfigMap
    apply.go:283: 2025-10-20T16:20:23.165324317Z: Deleting mqtt-backend Deployment
    apply.go:283: 2025-10-20T16:20:23.169600967Z: Deleting mqtt-backend Service
    apply.go:283: 2025-10-20T16:20:23.195491099Z: Deleting gateway-tlsroute-terminate Gateway
    apply.go:283: 2025-10-20T16:20:23.212827969Z: Deleting gateway-conformance-mqtt-test TLSRoute
=== NAME  TestConformance
    apply.go:239: 2025-10-20T16:20:23.23080456Z: Deleting mismatch-ca-certificate
    apply.go:239: 2025-10-20T16:20:23.242068692Z: Deleting tls-terminate-checks-certificate
    apply.go:239: 2025-10-20T16:20:23.258569208Z: Deleting tls-checks-certificate
    apply.go:239: 2025-10-20T16:20:23.270796213Z: Deleting tls-checks-ca-certificate
    apply.go:239: 2025-10-20T16:20:23.282990765Z: Deleting tls-passthrough-checks-certificate
    apply.go:239: 2025-10-20T16:20:23.292977393Z: Deleting tls-passthrough-checks-certificate
    apply.go:239: 2025-10-20T16:20:23.30875932Z: Deleting tls-validity-checks-certificate
    apply.go:239: 2025-10-20T16:20:23.32224744Z: Deleting certificate
    apply.go:283: 2025-10-20T16:20:23.346802631Z: Deleting coredns ConfigMap
    apply.go:283: 2025-10-20T16:20:23.359017617Z: Deleting coredns Deployment
    apply.go:283: 2025-10-20T16:20:23.378899715Z: Deleting coredns Service
    apply.go:283: 2025-10-20T16:20:23.423350078Z: Deleting grpc-infra-backend-v3 Deployment
    apply.go:283: 2025-10-20T16:20:23.440036553Z: Deleting grpc-infra-backend-v3 Service
    apply.go:283: 2025-10-20T16:20:23.470856983Z: Deleting grpc-infra-backend-v2 Deployment
    apply.go:283: 2025-10-20T16:20:23.487899446Z: Deleting grpc-infra-backend-v2 Service
    apply.go:283: 2025-10-20T16:20:23.523779145Z: Deleting grpc-infra-backend-v1 Deployment
    apply.go:283: 2025-10-20T16:20:23.545661933Z: Deleting grpc-infra-backend-v1 Service
    apply.go:283: 2025-10-20T16:20:23.569287039Z: Deleting web-backend Deployment
    apply.go:283: 2025-10-20T16:20:23.575834474Z: Deleting web-backend Service
    apply.go:283: 2025-10-20T16:20:23.594723903Z: Deleting gateway-conformance-web-backend Namespace
    apply.go:283: 2025-10-20T16:20:23.60211597Z: Deleting app-backend-v2 Deployment
    apply.go:283: 2025-10-20T16:20:23.608994287Z: Deleting app-backend-v2 Service
    apply.go:283: 2025-10-20T16:20:23.654774806Z: Deleting app-backend-v1 Deployment
    apply.go:283: 2025-10-20T16:20:23.679450296Z: Deleting app-backend-v1 Service
    apply.go:283: 2025-10-20T16:20:23.709816551Z: Deleting tls-backend Deployment
    apply.go:283: 2025-10-20T16:20:23.719451715Z: Deleting tls-backend Service
    apply.go:283: 2025-10-20T16:20:23.744768421Z: Deleting gateway-conformance-app-backend Namespace
    apply.go:283: 2025-10-20T16:20:23.757067263Z: Deleting tls-backend Deployment
    apply.go:283: 2025-10-20T16:20:23.768259503Z: Deleting tls-backend Service
    apply.go:283: 2025-10-20T16:20:23.803475947Z: Deleting infra-backend-v3 Deployment
    apply.go:283: 2025-10-20T16:20:23.811883727Z: Deleting infra-backend-v3 Service
    apply.go:283: 2025-10-20T16:20:23.829847393Z: Deleting infra-backend-v2 Deployment
    apply.go:283: 2025-10-20T16:20:23.84122509Z: Deleting infra-backend-v2 Service
    apply.go:283: 2025-10-20T16:20:23.867631498Z: Deleting infra-backend-v1 Deployment
    apply.go:283: 2025-10-20T16:20:23.875503861Z: Deleting infra-backend-v1 Service
    apply.go:283: 2025-10-20T16:20:23.900993105Z: Deleting backend-namespaces Gateway
    apply.go:283: 2025-10-20T16:20:23.921237923Z: Deleting all-namespaces Gateway
    apply.go:283: 2025-10-20T16:20:23.930512185Z: Deleting same-namespace-with-https-listener Gateway
    apply.go:283: 2025-10-20T16:20:24.019609701Z: Deleting same-namespace Gateway
    apply.go:283: 2025-10-20T16:20:24.036991554Z: Deleting gateway-conformance-infra Namespace
--- PASS: TestConformance (64.67s)
    --- PASS: TestConformance/TLSRouteTerminateSimpleSameNamespace (41.19s)
        --- PASS: TestConformance/TLSRouteTerminateSimpleSameNamespace/Simple_MQTT_TLS_request_matching_TLSRoute_should_reach_mqtt-backend (30.03s)
PASS
ok      sigs.k8s.io/gateway-api/conformance     64.686s

Which issue(s) this PR fixes: Relates #3466

Does this PR introduce a user-facing change?:

NONE

phuhung273 avatar Sep 29 '25 15:09 phuhung273

Welcome @phuhung273!

It looks like this is your first PR to kubernetes-sigs/gateway-api 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes-sigs/gateway-api has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. :smiley:

k8s-ci-robot avatar Sep 29 '25 15:09 k8s-ci-robot

Hi @phuhung273. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

k8s-ci-robot avatar Sep 29 '25 15:09 k8s-ci-robot

/ok-to-test

I'm not quite sure if this is specified explicitly (I don't see it mentioned in https://gateway-api.sigs.k8s.io/reference/spec/#listenertlsconfig or https://gateway-api.sigs.k8s.io/geps/gep-2907), but is mode: Terminate with TLSRoute a core conformance expectation now, or extended conformance and thus should have a new, specific feature flag like features.SupportTLSRouteModeTerminate?

mikemorris avatar Sep 29 '25 21:09 mikemorris

is mode: Terminate with TLSRoute a core conformance expectation now, or extended ?

Thanks for taking a look @mikemorris. I'm not sure about that, but can see we have a current features.SupportTLSRoute https://github.com/kubernetes-sigs/gateway-api/blob/530c1ee46e0d851d369810e4500f0339c40d8aa1/conformance/tests/tlsroute-simple-same-namespace.go#L45

phuhung273 avatar Sep 29 '25 23:09 phuhung273

Yeah, this absolutely should have a new feature name, so that implementations can support as they are ready to.

youngnick avatar Sep 30 '25 03:09 youngnick

@phuhung273, thanks for getting us started!

Also, while it's valid to use HTTP as the inner protocol, we should also end up testing bare TCP functions as well.

youngnick avatar Sep 30 '25 03:09 youngnick

Also, while it's valid to use HTTP as the inner protocol, we should also end up testing bare TCP functions as well.

Thank you also for taking a look @youngnick. Absolutely i will try this (although having no idea what youre saying currently 😅)

Right now Im just trying to complete a simple case. This one seems useful https://github.com/projectcontour/contour/blob/main/internal/featuretests/v3/tlsroute_test.go, im trying to replicate the same.

phuhung273 avatar Sep 30 '25 04:09 phuhung273

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: phuhung273 Once this PR has been reviewed and has the lgtm label, please assign danwinship for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot avatar Sep 30 '25 08:09 k8s-ci-robot

Verified with Contour, please see PR description for test output. Also added features.SupportTLSRouteModeTerminate as discussed. Let me know what you think @mikemorris @youngnick. Thank you so much.

phuhung273 avatar Sep 30 '25 10:09 phuhung273

So, just clarifying: per our TLS Guide we have the following supports and cases:

  • TLS = Passthrough + TLSRoute = Passthrough
  • TLS = Terminate + TLSRoute = Not supported (and you should be achieving this with HTTPRoute?)
  • TLS = Passthrough + TCPRoute = Not supported (you need to do it with PT + TLSRoute)
  • TLS = Terminate + TCPRoute = TLS offload (supported)
  • TLS = Terminate + HTTP/GRPCRoutes = supported

I am wondering why we are considering a TLS = Terminate + TLSRoute here? Is this just an alternative to TLS = Terminate + TCPRoute?

I think in this case it may be a bit misleading on which route I want / should use, if 2 do the same job.

Also, we are explicitly saying on the TLSRoute GEP that we don't support TLSRoute termination (https://github.com/kubernetes-sigs/gateway-api/pull/4064/files#diff-7e6544694a096dc122ce2ef4d38e4a47bfe14b72d5ae3603af9c17f6ccf23339R33) so if we can first agree on the GEP on if we should or not, then move to Conformance I would appreciate for my own sanity :sweat_smile:

Thanks!

rikatz avatar Sep 30 '25 12:09 rikatz

Ok can see this table in the guide image

Thanks @rikatz for the update. I will wait for GEP-2643 to finalize.

But currently we don't have any conformance for TCPRoute in Terminate mode. So I can add one rite ?

phuhung273 avatar Sep 30 '25 12:09 phuhung273

@rikatz TLSRoute support for attaching to Gateway listeners with mode: Terminate was added in https://github.com/kubernetes-sigs/gateway-api/pull/3458. More detail on the use cases for this combination can be found in https://github.com/kubernetes-sigs/gateway-api/issues/2111 and https://github.com/kubernetes-sigs/gateway-api/pull/3458#discussion_r1935423388.

It sounds like we may need to resolve some inconsistent documentation as mentioned in https://github.com/kubernetes-sigs/gateway-api/issues/1474?

mikemorris avatar Sep 30 '25 13:09 mikemorris

thanks Mike. I have missed those, or maybe and inconsistently left them behind. Will take a look on them, but I am wondering if it would be good/proper that we have all of this mapped on the GEP before moving with more conformance that may not reflect the final state of the GEP

rikatz avatar Sep 30 '25 13:09 rikatz

wondering if it would be good/proper that we have all of this mapped on the GEP before moving with more conformance that may not reflect the final state of the GEP

We've been somewhat inconsistent about this, but we generally haven't enforced substantial retroactive edits to older GEPs, instead allowing newer GEPs to supercede and prioritizing conformance tests and docs reflecting the current state while allowing older GEPs to stay as historical documents.

mikemorris avatar Sep 30 '25 16:09 mikemorris

yeah but in this case we don't have a TLSRoute GEP at all, and my plan is to have some covering all of the features/conformance that are already in place for TLSRoute

rikatz avatar Sep 30 '25 16:09 rikatz

Updated the GEP proposal to add TLSRoute termination: https://github.com/kubernetes-sigs/gateway-api/pull/4064/commits/23c275e1b508cb9bea1108c16a7f7f9b36341482

rikatz avatar Sep 30 '25 18:09 rikatz

If it's not covered in that summary table, and we have no GEP mentioning it, then we can't just change the docs and call it done. We can add it to the TLSRoute GEP as a new area, but it then needs to be reviewed and debated. Right now I don't see the use case for TLSRoute termination.

candita avatar Oct 03 '25 18:10 candita

TLSRoute support for listeners with mode: Terminate already went through review previously and was added in https://github.com/kubernetes-sigs/gateway-api/pull/3458 as an update to GEP-2907, @rikatz latest updates are just ensuring for completeness this functionality is also covered within the complete TLSRoute GEP.

mikemorris avatar Oct 06 '25 23:10 mikemorris

TLSRoute support for listeners with mode: Terminate already went through review previously and was added in #3458 as an update to GEP-2907, @rikatz latest updates are just ensuring for completeness this functionality is also covered within the complete TLSRoute GEP.

The GEP it was added to was a Memorandum GEP and not highly scrutinized. TLSRoute support for Terminate mode may eventually make it in as Extended, but we agreed in our last community meeting that we should not be using Memorandum GEPs to publish new features.

candita avatar Oct 09 '25 17:10 candita

/cc @rostislavbobo @kl52752

robscott avatar Oct 17 '25 23:10 robscott

just a side note, I still wanna get https://github.com/kubernetes-sigs/gateway-api/pull/4064 merged before any new conformance test :) I will be working on updates on it this week, but would appreciate anyone's review on it once more.

rikatz avatar Oct 20 '25 12:10 rikatz

What I would really like to see is a bit of user documentation that illustrates when it is appropriate to use a TLSRoute vs HTTPRoute to terminate. Is it just when you don't have a hostname that you would terminate with a TLSRoute?

candita avatar Nov 26 '25 00:11 candita

What I would really like to see is a bit of user documentation that illustrates when it is appropriate to use a TLSRoute vs HTTPRoute to terminate.

I think these 2 comments in GEP-2643 well explain the comparison:

  • https://github.com/kubernetes-sigs/gateway-api/pull/4064#discussion_r2564957146
  • https://github.com/kubernetes-sigs/gateway-api/pull/4064#discussion_r2565147158

Lets wait for GEP-2643 merged. After that, if there is no end-user doc, I can open another doc PR before this conformance goes in.

phuhung273 avatar Nov 28 '25 05:11 phuhung273

What I would really like to see is a bit of user documentation that illustrates when it is appropriate to use a TLSRoute vs HTTPRoute to terminate.

@candita @phuhung273 have you seen https://github.com/kubernetes-sigs/gateway-api/pull/4330? Hopefully this could be a good start to better explain this!

mikemorris avatar Dec 11 '25 20:12 mikemorris

@phuhung273 I am planning on reviewing the PR this week, do you mind rebasing it? (we did a bunch of changes to move conformance dependencies out of main code, so you can use the mqtt library also without concerns!)

Thanks!

rikatz avatar Dec 15 '25 19:12 rikatz

Thanks for reminding @rikatz and the conformance dependencies structure change, I've rebased. Good to see GEP-2643 finalized.

phuhung273 avatar Dec 15 '25 23:12 phuhung273

/cc

rikatz avatar Dec 16 '25 12:12 rikatz

/assign

rikatz avatar Dec 16 '25 12:12 rikatz

@rikatz not sure why I cannot comment on your feedback regarding MQTT image. But agree it is better not requiring user to pull too many things.

phuhung273 avatar Dec 17 '25 14:12 phuhung273

@phuhung273 a question, were you able to test this with some implementation?

I am testing with kgateway and failing, but I also did tested with kgateway and a simple TCP service and it is failing.

@davidjumani do you know if kgateway already supports TLSRoute on termination?

Thanks!

rikatz avatar Dec 18 '25 14:12 rikatz