Conformance tests cannot run on v1.2.1

[snehachhabria]

This is caused by https://github.com/kubernetes-sigs/gateway-api/pull/3211 which overrides the passed in Client as mentioned here: https://github.com/kubernetes-sigs/gateway-api/pull/3343#discussion_r1764102336. Due to this change Microsoft Azure's implementation of the conformance tests is broken because we had a custom Client that was passed in while setting up the conformance suite.

What would you like to be added: Ensure that the Client is still configurable for implementors to override if needed

Why this is needed: Unable to upgrade to v1.2.1 because it breaks conformance tests.

cc @howardjohn @BobyMCbobs

[robscott]

Thanks for reporting this @snehachhabria! To clarify, is this a regression between v1.2.0 and v1.2.1, or is it a problem with all v1.2 releases?

/cc @mlavacca

[snehachhabria]

Thanks for reporting this @snehachhabria! To clarify, is this a regression between v1.2.0 and v1.2.1, or is it a problem with all v1.2 releases?

/cc @mlavacca it's a problem with all v1.2 releases

[robscott]

/kind bug

[mlavacca]

Thanks for reporting this issue, @snehachhabria! With #3343 we introduced the possibility to customize the client options to be able to pass scheme, mapper, etc. Isn't the ClientOptions field enough to fit your needs?

[snehachhabria]

Thanks for reporting this issue, @snehachhabria! With #3343 we introduced the possibility to customize the client options to be able to pass scheme, mapper, etc. Isn't the ClientOptions field enough to fit your needs?

Unfortunately not, exposing the Client allows us to modify the writer interface for the Create/Update function to add some custom annotations while creating gateway objects

[youngnick]

Are there any more details you can share about what you need to add? Maybe that might be useful for other folks as well, and we should build support directly into the tests?

[snehachhabria]

Our implementation requires custom annotations on the gateway object, sample can be found here: https://learn.microsoft.com/en-us/azure/application-gateway/for-containers/how-to-traffic-splitting-gateway-api?tabs=alb-managed#deploy-the-required-gateway-api-resources . In order to add these annotations to the gateway objects created by the conformance suite we have modified the Create/Update function exposed via the client.

It would be great if support for annotations on Gateway objects can be supported directly via the test suite, similar to how namespace annotations are exposed: https://github.com/kubernetes-sigs/gateway-api/blob/99a3934c6bc1ce0874f3a4c5f20cafd8977ffcb4/conformance/utils/suite/suite.go#L137 . That way it could potentially be used by other implementors as well

[youngnick]

To be honest, it's really intended that what's described in that doc is implemented by having a GatewayClass that sets all Gateways associated to it to use the configured ALB.

I really don't like the approach of using annotations on Gateway to do this - it's totally against one of the major design goals of Gateway API, which is to pull that kind of config out of annotations and into structured fields and more specific resources.

[robscott]

+1 to @youngnick's comment here. In my mind, one of the most important purposes of conformance tests is ensuring that users will get the same shared experience out of the box with our portable features. Obviously implementations should feel welcome to add features on top of the core API, but we really want the base API and examples to work without any changes (other than setting a GatewayClass name on a Gateway).

If I'm understanding this specific modification correctly, it seems like users would need to manually configure some annotations on each Gateway, which goes against the API experience we're trying to guarantee with conformance tests and status.

[snehachhabria]

@youngnick and @robscott thanks a lot of sharing your thoughts. I completely understand the primary design goal of Gateway API is not to have any configurability via annotations. Given the limitations that existed at the time we were implementing Gateway API and our infrastructure/design needs we had to go down the route of using an annotation on the Gateway object.

I understand if providing support for annotations on the gateway object in the conformance is against the experience desired and shouldn't be exposed. But if we could go back to having the Client exposed, like it was prior to release v1.2, we would still be able to run the conformance tests and generate the reports. We are still conformant with all of Gateway API's requirements and would like to continue doing so, the conformance tests aid this and it would be great if we can run them as is without too many modifications from our end.

[snehachhabria]

@youngnick and @robscott thanks a lot of sharing your thoughts. I completely understand the primary design goal of Gateway API is not to have any configurability via annotations. Given the limitations that existed at the time we were implementing Gateway API and our infrastructure/design needs we had to go down the route of using an annotation on the Gateway object.

I understand if providing support for annotations on the gateway object in the conformance is against the experience desired and shouldn't be exposed. But if we could go back to having the Client exposed, like it was prior to release v1.2, we would still be able to run the conformance tests and generate the reports. We are still conformant with all of Gateway API's requirements and would like to continue doing so, the conformance tests aid this and it would be great if we can run them as is without too many modifications from our end.

@youngnick and @robscott any thoughts on this? it would be great if the Client could be exposed back like it was prior to 1.2 release

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle rotten
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue with /reopen
• Mark this issue as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

[k8s-ci-robot]

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue with /reopen
• Mark this issue as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.