gateway-api icon indicating copy to clipboard operation
gateway-api copied to clipboard

Incorrect conformance tests and reporting for Mesh?

Open LiorLieberman opened this issue 9 months ago • 6 comments

What happened: Looking into our conformance reports and tests for Mesh, I see features being reported as "supported" but suspect we are not even testing them. Taking istio as an example.

  1. The last conformance report reports support for many features.
  2. If we take HTTPRoutePathRewrite as an example, we can see that the test for this feature only deploys an HTTPRoute with a Gateway as a parentRef and not a Service.
  3. In fact, most of the tests for HTTPRoute features are not covering mesh functionalities. There are only a few tests that actually tests mesh and those are the feature names that start with "Mesh"
  4. This leads to incorrect reporting - implementations that support N/W HTTPRoute functionalities reports the same support for mesh if they pick the mesh profile.
  5. While this case should be ideal, its not 100% the case, and we basically have minimal coverage for Mesh support.

What you expected to happen: We should test every feature thats applicable to mesh (should be most of the features if we stick to the vision) with Service as a parentRef in addition to Gateway.

Ideally we should address this as a framework change with support to exclude features as opposed to add this functionality to every test. The Service-as-a-parentRef tests should only be running if the implementation indicated that they support the MESH profile.

/area conformance /cc @howardjohn @robscott @youngnick @mlavacca @kflynn @shaneutt @mikemorris

LiorLieberman avatar Feb 03 '25 01:02 LiorLieberman