kubernetes-ingress-controller
kubernetes-ingress-controller copied to clipboard
Published
20 hours ago •
Kong
Kong
Integration test `TestTCPRouteEssentials` fails
Problem statement
Recently changed TestTCPRouteEssentials failed on an unrelated PR CI run:
=== RUN TestTCPRouteEssentials/essentials/verifying_behavior_when_Gateway_is_deleted_and_recreated
tcproute_test.go:240: deleting the GatewayClass
tcproute_test.go:243: verifying that the Gateway gets unlinked from the route via status
tcproute_test.go:247: verifying that the data-plane configuration from the TCPRoute gets dropped with the GatewayClass now removed
tcproute_test.go:250: putting the GatewayClass back
tcproute_test.go:254: verifying that the Gateway gets linked to the route via status
tcproute_test.go:258: verifying that creating the GatewayClass again triggers reconciliation of TCPRoutes and the route becomes available again
I0328 11:40:43.461860 15404 utils.go:29] "TCPRoute has been configured on the data-plane" logger="controllers.TCPRoute" GatewayV1Alpha2TCPRoute="ns-cff-ff3/b27dc3c2-311e-4f18-8d74-63d1f497402e" namespace="ns-cff-ff3" name="b27dc3c2-311e-4f18-8d74-63d1f497402e"
I0328 11:40:45.486142 15404 utils.go:29] "TCPRoute has been configured on the data-plane" logger="controllers.TCPRoute" GatewayV1Alpha2TCPRoute="ns-cff-ff3/b27dc3c2-311e-4f18-8d74-63d1f497402e" namespace="ns-cff-ff3" name="b27dc3c2-311e-4f18-8d74-63d1f497402e"
tcproute_test.go:259:
Error Trace: /home/runner/work/kubernetes-ingress-controller/kubernetes-ingress-controller/test/integration/isolated/assertion_helpers.go:43
/opt/hostedtoolcache/go/1.21.6/x64/src/runtime/asm_amd64.s:1650
Error: Received unexpected error:
read tcp 172.18.0.1:36566->172.18.128.3:8888: read: connection reset by peer
tcproute_test.go:259:
Error Trace: /home/runner/work/kubernetes-ingress-controller/kubernetes-ingress-controller/test/integration/isolated/assertion_helpers.go:42
/home/runner/work/kubernetes-ingress-controller/kubernetes-ingress-controller/test/integration/isolated/tcproute_test.go:259
/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/env/env.go:435
/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/env/env.go:468
Error: Condition never satisfied
Test: TestTCPRouteEssentials/essentials/verifying_behavior_when_Gateway_is_deleted_and_recreated
tcproute_test.go:261: deleting the Gateway
tcproute_test.go:264: verifying that the Gateway gets unlinked from the route via status
tcproute_test.go:268: verifying that the data-plane configuration from the TCPRoute gets dropped with the Gateway now removed
tcproute_test.go:271: putting the Gateway back
tcproute_test.go:282: verifying that the Gateway gets linked to the route via status
I0328 11:43:43.978469 15404 utils.go:29] "Marking gateway as accepted" logger="controllers.Gateway" GatewayV1Gateway="ns-cff-ff3/8f76848a-cb1c-4b2e-b85c-10f2138d8da9" namespace="ns-cff-ff3" name="8f76848a-cb1c-4b2e-b85c-10f2138d8da9"
I0328 11:43:44.037657 15404 utils.go:29] "TCPRoute has been configured on the data-plane" logger="controllers.TCPRoute" GatewayV1Alpha2TCPRoute="ns-cff-ff3/b27dc3c2-311e-4f18-8d74-63d1f497402e" namespace="ns-cff-ff3" name="b27dc3c2-311e-4f18-8d74-63d1f497402e"
I0328 11:43:44.081030 15404 utils.go:29] "TCPRoute has been configured on the data-plane" logger="controllers.TCPRoute" GatewayV1Alpha2TCPRoute="ns-cff-ff3/b27dc3c2-311e-4f18-8d74-63d1f497402e" namespace="ns-cff-ff3" name="b27dc3c2-311e-4f18-8d74-63d1f497402e"
tcproute_test.go:286: verifying that creating the Gateway again triggers reconciliation of TCPRoutes and the route becomes available again
tcproute_test.go:287:
Error Trace: /home/runner/work/kubernetes-ingress-controller/kubernetes-ingress-controller/test/integration/isolated/assertion_helpers.go:43
/opt/hostedtoolcache/go/1.21.6/x64/src/runtime/asm_amd64.s:1650
Error: Received unexpected error:
read tcp 172.18.0.1:38584->172.18.128.3:8888: read: connection reset by peer
tcproute_test.go:287:
Error Trace: /home/runner/work/kubernetes-ingress-controller/kubernetes-ingress-controller/test/integration/isolated/assertion_helpers.go:42
/home/runner/work/kubernetes-ingress-controller/kubernetes-ingress-controller/test/integration/isolated/tcproute_test.go:287
/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/env/env.go:435
/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/env/env.go:468
Error: Condition never satisfied
Test: TestTCPRouteEssentials/essentials/verifying_behavior_when_Gateway_is_deleted_and_recreated
tcproute_test.go:289: deleting both GatewayClass and Gateway rapidly
tcproute_test.go:293: verifying that the Gateway gets unlinked from the route via status
E0328 11:46:44.240285 15404 gateway_controller.go:197] "Could not retrieve gatewayclass" err="GatewayClass.gateway.networking.k8s.io \"a904a064-64fd-4369-bcf1-61c16c63920b\" not found" logger="controllers.Gateway" gatewayclass="a904a064-64fd-4369-bcf1-61c16c63920b"
tcproute_test.go:297: verifying that the data-plane configuration from the TCPRoute does not get orphaned with the GatewayClass and Gateway gone
tcproute_test.go:300: putting the GatewayClass back
tcproute_test.go:304: putting the Gateway back
tcproute_test.go:315: verifying that the Gateway gets linked to the route via status
I0328 11:46:44.772329 15404 utils.go:29] "Marking gateway as accepted" logger="controllers.Gateway" GatewayV1Gateway="ns-cff-ff3/8f76848a-cb1c-4b2e-b85c-10f2138d8da9" namespace="ns-cff-ff3" name="8f76848a-cb1c-4b2e-b85c-10f2138d8da9"
I0328 11:46:44.833257 15404 utils.go:29] "TCPRoute has been configured on the data-plane" logger="controllers.TCPRoute" GatewayV1Alpha2TCPRoute="ns-cff-ff3/b27dc3c2-311e-4f18-8d74-63d1f497402e" namespace="ns-cff-ff3" name="b27dc3c2-311e-4f18-8d74-63d1f497402e"
I0328 11:46:44.949636 15404 utils.go:29] "TCPRoute has been configured on the data-plane" logger="controllers.TCPRoute" GatewayV1Alpha2TCPRoute="ns-cff-ff3/b27dc3c2-311e-4f18-8d74-63d1f497402e" namespace="ns-cff-ff3" name="b27dc3c2-311e-4f18-8d74-63d1f497402e"
tcproute_test.go:319: verifying that creating the Gateway again triggers reconciliation of TCPRoutes and the route becomes available again
tcproute_test.go:320:
Error Trace: /home/runner/work/kubernetes-ingress-controller/kubernetes-ingress-controller/test/integration/isolated/assertion_helpers.go:43
/opt/hostedtoolcache/go/1.21.6/x64/src/runtime/asm_amd64.s:1650
Error: Received unexpected error:
read tcp 172.18.0.1:58578->172.18.128.3:8888: read: connection reset by peer
tcproute_test.go:320:
Error Trace: /home/runner/work/kubernetes-ingress-controller/kubernetes-ingress-controller/test/integration/isolated/assertion_helpers.go:42
/home/runner/work/kubernetes-ingress-controller/kubernetes-ingress-controller/test/integration/isolated/tcproute_test.go:320
/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/env/env.go:435
/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/env/env.go:468
Error: Condition never satisfied
Test: TestTCPRouteEssentials/essentials/verifying_behavior_when_Gateway_is_deleted_and_recreated
https://github.com/Kong/kubernetes-ingress-controller/actions/runs/8466614681/job/23195804013?pr=5765
The above seems like a very random GH Action hiccup. But there is a more common flake for this test case
=== RUN TestTCPRouteEssentials/essentials/verifying_behavior_when_TCPRoute_is_modified
tcproute_test.go:187: removing the parentrefs from the TCPRoute
tcproute_test.go:202: verifying that the Gateway gets unlinked from the route via status
tcproute_test.go:206: verifying that the tcpecho is no longer responding
tcproute_test.go:216: putting the parentRefs back
tcproute_test.go:225: verifying that the Gateway gets linked to the route via status
tcproute_test.go:227:
Error Trace: /home/runner/work/kubernetes-ingress-controller/kubernetes-ingress-controller/test/integration/isolated/tcproute_test.go:227
/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/env/env.go:435
/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/env/env.go:468
Error: Condition never satisfied
Test: TestTCPRouteEssentials/essentials/verifying_behavior_when_TCPRoute_is_modified
tcproute_test.go:229: verifying that putting the parentRefs back results in the routes becoming available again
tcproute_test.go:210: no longer responding check failure state: eof=false, reset=false, err=<nil>
