kubernetes-sigs
Fix conformance test flags are exported to other libraries that use the conformance test suite
What type of PR is this?
What this PR does / why we need it:
/kind bug /area conformance
Which issue(s) this PR fixes:
Fixes https://github.com/kubernetes-sigs/gateway-api/issues/2841.
Currently when implementing GEP-2162, the features need to be imported. However, in the test suite, this import results in the variation of flags within our production code. The goal is to allow the importation of features without modifying the product code.
Does this PR introduce a user-facing change?:
Welcome @hanxiaop!
It looks like this is your first PR to kubernetes-sigs/gateway-api 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.
You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.
You can also check if kubernetes-sigs/gateway-api has its own contribution guidelines.
You may want to refer to our testing guide if you run into trouble with your tests not passing.
If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!
Thank you, and welcome to Kubernetes. :smiley:
Hi @hanxiaop. Thanks for your PR.
I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.
Once the patch is verified, the new status will be reflected by the ok-to-test label.
I understand the commands that are listed here.
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.
Also cc @LiorLieberman to take a look, the feature was introducted in https://github.com/istio/istio/pull/48770.
Thanks @hanxiaop! Can you elaborate on the problem? I am not sure I understand
Thanks @hanxiaop! Can you elaborate on the problem? I am not sure I understand
@LiorLieberman So basically, in that PR, you imported the test suite code into the product code:
https://github.com/istio/istio/blob/42572fab5be79af8de284953ec72cbf439631da8/pilot/pkg/config/kube/gateway/supported_features.go#L24,
and after https://github.com/kubernetes-sigs/gateway-api/pull/2801/files#diff-3532d0dde2d46785ab138cfe0fa7bddc5c1901dde4fe4079880451c010ea8254R34, the flag package was also imported into the suite. After that, our command-line tool was filled with gateway conformance test flags alongside our own pflags, since the test flags were imported in the dependency and thus in the flag set. This PR moves the features file out of the suite, so that we can directly import the features instead of the entire suite package and avoid issues like flags being altered.
Thanks @hanxiaop! Can you elaborate on the problem? I am not sure I understand
@LiorLieberman So basically, in that PR, you imported the test suite code into the product code:
https://github.com/istio/istio/blob/42572fab5be79af8de284953ec72cbf439631da8/pilot/pkg/config/kube/gateway/supported_features.go#L24,
and after https://github.com/kubernetes-sigs/gateway-api/pull/2801/files#diff-3532d0dde2d46785ab138cfe0fa7bddc5c1901dde4fe4079880451c010ea8254R34, the flag package was also imported into the suite. After that, our command-line tool was filled with gateway conformance test flags alongside our own pflags, since the test flags were imported in the dependency and thus in the flag set. This PR moves the features file out of the suite, so that we can directly import the features instead of the entire suite package and avoid issues like flags being altered.
Thanks, I think many implementations uses suite to import features. This would require all of them to move to features or their code wont work.
Not sure whats the procedure in such case and whether it is considered breaking change or not. /cc @youngnick @shaneutt @robscott
Thanks, I think many implementations uses suite to import features. This would require all of them to move to features or their code wont work.
If we really want to maintain Go-level compatibility (which I think is a non-goal of the project, but I could be wrong) we can always add a type alias in suite.
Not sure whats the procedure in such case and whether it is considered breaking change or not. /cc @youngnick @shaneutt @robscott
I think this will not be an issue or a breaking change, as it's fairly common for bumping a dependency to result in a change in the package path.
This looks good to me, @hanxiaop! Would you mind if we wait for #2868 to land before merging this one (that's part of the conformance suite epic and is required for 1.1)?
@mlavacca no problem, I'll rebase once it's merged, thanks for the review!
Thanks @hanxiaop! Can you elaborate on the problem? I am not sure I understand
@LiorLieberman So basically, in that PR, you imported the test suite code into the product code: https://github.com/istio/istio/blob/42572fab5be79af8de284953ec72cbf439631da8/pilot/pkg/config/kube/gateway/supported_features.go#L24, and after https://github.com/kubernetes-sigs/gateway-api/pull/2801/files#diff-3532d0dde2d46785ab138cfe0fa7bddc5c1901dde4fe4079880451c010ea8254R34, the flag package was also imported into the suite. After that, our command-line tool was filled with gateway conformance test flags alongside our own pflags, since the test flags were imported in the dependency and thus in the flag set. This PR moves the features file out of the suite, so that we can directly import the features instead of the entire suite package and avoid issues like flags being altered.
Thanks, I think many implementations uses
suiteto import features. This would require all of them to move tofeaturesor their code wont work.Not sure whats the procedure in such case and whether it is considered breaking change or not. /cc @youngnick @shaneutt @robscott
Many changes have been made to the suite for its graduation; implementations will have to adapt to those changes in any case when bumping to the next Gateway API version. I do not see this kind of change as problematic.
We need to change all the references to the features in the actual tests, under
conformance/tests. As it is, the code does not even compile.
@mlavacca Fixed, missed that dependency. It would be great if someone could add the ok-to-test label so I can check if there are more things being broken.
@mlavacca @youngnick @shaneutt @robscott @howardjohn @keithmattix PTAL, thanks!
[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: hanxiaop, howardjohn, robscott
The full list of commands accepted by this bot can be found here.
The pull request process is described here
- ~~OWNERS~~ [robscott]
Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment
@hanxiaop if you'd prefer we can use #2980 which copied this PR and made some modifications to pass presubmits.
@hanxiaop if you'd prefer we can use #2980 which copied this PR and made some modifications to pass presubmits.
@robscott Thanks! I think it should be good now. New tests are continually being added to the folder, so I need to do a rebase again. If it doesn't work I think we can use that PR.