gateway icon indicating copy to clipboard operation
gateway copied to clipboard

Initial Gateway API Conformance

Open danehans opened this issue 3 years ago • 7 comments

The community has established Gateway API conformance as the target of the 0.0.2 release. This is a meta issue, in that issues, PRs, etc. that target this goal should be linked to this issue.

danehans avatar May 13 '22 18:05 danehans

Since conformance tests are a moving target and being fully conformant may be a considerable undertaking, we should identify a subset of tests that are the v0.2.0 target.

xref: https://github.com/envoyproxy/gateway/issues/11#issuecomment-1126413344 xref conformance tests: https://github.com/kubernetes-sigs/gateway-api/tree/master/conformance

@youngnick feel free to provide input.

danehans avatar May 13 '22 20:05 danehans

Yes, I think we should pick a Gateway API release as our first milestone target, we could choose either v0.4.3, the final v1alpha2 release, or the forthcoming v0.5.0, the first v1beta1 release. I'm inclined toward the latter.

youngnick avatar May 17 '22 10:05 youngnick

@skasisom6 the v0.2.0 goal is for Envoy Gateway to pass Gateway API conformance tests. This means that Envoy Gateway will provide a set of core API Gateway functionality. When you have a moment, please review Gateway API conformance, along with the conformance tests to learn more about conformance. Let us know if these features are not consistent with your requirements for the release.

danehans avatar May 17 '22 18:05 danehans

@skasisom6 the v0.2.0 goal is for Envoy Gateway to pass Gateway API conformance tests. This means that Envoy Gateway will provide a set of core API Gateway functionality. When you have a moment, please review Gateway API conformance, along with the conformance tests to learn more about conformance. Let us know if these features are not consistent with your requirements for the release.

danehans avatar May 17 '22 18:05 danehans

I can work on getting the Gateway API conformance tests running in CI so as soon as we have something integrated, we can start running tests against it. I'll file a specific issue for it.

skriss avatar Jul 15 '22 17:07 skriss

v0.2.0 will focus on httproute conformance, e.g. not tlsroute, tcproute, etc. @kflynn we should make sure the docs state this.

xref https://github.com/envoyproxy/gateway/issues/17

danehans avatar Aug 16 '22 22:08 danehans

Large blockers for getting Gateway API conformance tests to run:

  • [x] https://github.com/envoyproxy/gateway/issues/330 (conformance uses the address in Gateway.Status.Addresses to send HTTP requests; we need to set this to the provisioned Service's external IP) (PR: https://github.com/envoyproxy/gateway/pull/352)
  • [x] https://github.com/envoyproxy/gateway/issues/345 (conformance tests won't proceed until a Gateway has a Condition of "Ready: true") (PR: https://github.com/envoyproxy/gateway/pull/392)
  • [x] https://github.com/envoyproxy/gateway/issues/329 (conformance is equally concerned with HTTP requests succeeding and Gateway API resources having the correct statuses; we need to be writing statuses to the apiserver)
  • [x] https://github.com/envoyproxy/gateway/issues/349 (EG's model is to merge all Gateways of a GatewayClass but the base conformance Gateways are not mergeable)

Specific "core" conformance features still to be implemented:

  • [x] #159 (PR: https://github.com/envoyproxy/gateway/pull/249)
  • [x] #158 (PR: https://github.com/envoyproxy/gateway/pull/363)
  • [x] #162
  • [x] #168 (TLS passthrough with TLSRoute does look to be a core conformance feature, but there are not currently conformance tests for it) (PR: https://github.com/envoyproxy/gateway/pull/402)
  • [x] #411

It's although worth noting that the Gateway API conformance tests are still a work in progress -- not every feature tagged as "core" in the API spec has tests yet, but they are being continually added. So -- we should try to get all the core features implemented, whether they currently have a conformance test or not, so we're not behind the ball when corresponding tests do get added.

skriss avatar Sep 06 '22 21:09 skriss

Note that https://github.com/envoyproxy/gateway/issues/168 and https://github.com/envoyproxy/gateway/issues/411 are not required to pass conformance tests.

danehans avatar Oct 03 '22 23:10 danehans

2 remaining referencegrant tests should pass when #588 is fixed.

danehans avatar Oct 18 '22 21:10 danehans

Closing since all core conformance tests are passing. All but one RefernceGrant test (tracked by https://github.com/envoyproxy/gateway/issues/539) is passing.

danehans avatar Oct 19 '22 21:10 danehans