cert-manager-operator
cert-manager-operator copied to clipboard
openshift
CM-261: utilize ginkgo label filter to differentiate cloud platforms
Main changes:
- Bump onsi/gingko/v2 to v2.19.
- To make use of the newly introduced Label Sets feature. Benefits:
- It is more expressive and human-readable than regexp combining
--focusand--skip. - It extends the current
--label-filterto support filtering by key-value pairs, with the contains and not contains operations for the labels that have values. (more example usages can be found in the doc) - It gives more flexibility in filtering and sorting tests based on labels. As the test suite grows, we might apply more label types to identify Serial/Disruptive, Alpha/Beta, Conformance/MinVersion:xxx, etc.
- It is more expressive and human-readable than regexp combining
- To make use of the newly introduced Label Sets feature. Benefits:
- Add an env variable
E2E_GINKGO_LABEL_FILTERin the Makefile. Append-ginkgo.label-filter=$(E2E_GINKGO_LABEL_FILTER)to the go test command. - Apply labels
"Platform:AWS","Platform:GCP"and"Platform:IBM"to cases description accordingly. - Set the default value of
E2E_GINKGO_LABEL_FILTERin the Makefile to"Platform: isSubsetOf {AWS}".- It would then select and run both non-platform-specific and AWS-only cases.
Other changes:
- Remove some hardcoded skipping logic.
- Regroup ACME dns-01 cases by their provider platform. Each dns-01 provider now will have a dedicated
Context().
I've tested on the GCP cluster. It shows: (skipped 3 cases labeled "Platform:AWS", 1 labeled "Platform:IBM")
go test \
-timeout 1h \
-count 1 \
-v \
-p 1 \
-tags e2e \
-run "" \
./test/e2e \
-ginkgo.label-filter="Platform: isSubsetOf {GCP}"
...
Ran 17 of 21 Specs in 524.481 seconds
SUCCESS! -- 17 Passed | 0 Failed | 0 Pending | 4 Skipped
--- PASS: TestAll (524.49s)
PASS
ok github.com/openshift/cert-manager-operator/test/e2e 846.604s
@lunarwhite: This pull request references CM-261 which is a valid jira issue.
Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.17.0" version, but no target version was set.
In response to this:
Main changes:
- Bump onsi/gingko/v2 to v2.19.
- To make use of the newly introduced Label Sets feature. Benefits:
- It is more expressive and human-readable than regexp combining
--focusand--skip.- It extends the current
--label-filterto support filtering by key-value pairs, with the contains and not contains operations for the labels that have values. (more example usages can be found in the doc)- It gives more flexibility in filtering and sorting tests based on labels. As the test suite grows, we might apply more label types to identify Serial/Disruptive, Alpha/Beta, Conformance/MinVersion:xxx, etc.
- Add an env variable
E2E_GINKGO_LABEL_FILTERin the Makefile. Append-ginkgo.label-filter=$(E2E_GINKGO_LABEL_FILTER)to the go test command.- Apply labels
"Platform:AWS","Platform:GCP"and"Platform:IBM"to cases description accordingly.- Set the default value of
E2E_GINKGO_LABEL_FILTERin the Makefile to"Platform: isSubsetOf {AWS}".- It would then select and run both non-platform-specific and AWS-only cases.
Other changes:
- Remove some hardcoded skipping logic.
- Regroup ACME dns-01 cases by their provider platform. Each dns-01 provider now will have a dedicated
Context().I've tested on the GCP cluster. It shows: (skipped 3 cases labeled
"Platform:AWS", 1 labeled"Platform:IBM")go test \ -timeout 1h \ -count 1 \ -v \ -p 1 \ -tags e2e \ -run "" \ ./test/e2e \ -ginkgo.label-filter="Platform: isSubsetOf {GCP}" ... Ran 17 of 21 Specs in 524.481 seconds SUCCESS! -- 17 Passed | 0 Failed | 0 Pending | 4 Skipped --- PASS: TestAll (524.49s) PASS ok github.com/openshift/cert-manager-operator/test/e2e 846.604s
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.
/uncc deads2k /cc @swghosh @TrilokGeer
CI is green. Mark it as ready for review now.
To keep it simple and easy for review and updates, I created this separate PR for CM-261. I didn't touch the main logic of the existing cases, so your comments in #181 are not fully addressed. After this is done, we can rebase and update #181.
/lgtm /label docs-approved /label px-approved (non-user facing, only e2e test related changes)
[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: lunarwhite, swghosh
The full list of commands accepted by this bot can be found here.
The pull request process is described here
- ~~OWNERS~~ [swghosh]
Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment
![openshift-ci[bot] avatar](https://avatars.githubusercontent.com/in/91280?v=4 "Published by a pub.dev verified publisher"){kind=small}
@lunarwhite: This pull request references CM-261 which is a valid jira issue.
Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.18.0" version, but no target version was set.
In response to this:
Main changes:
- Bump onsi/gingko/v2 to v2.19.
- To make use of the newly introduced Label Sets feature. Benefits:
- It is more expressive and human-readable than regexp combining
--focusand--skip.- It extends the current
--label-filterto support filtering by key-value pairs, with the contains and not contains operations for the labels that have values. (more example usages can be found in the doc)- It gives more flexibility in filtering and sorting tests based on labels. As the test suite grows, we might apply more label types to identify Serial/Disruptive, Alpha/Beta, Conformance/MinVersion:xxx, etc.
- Add an env variable
E2E_GINKGO_LABEL_FILTERin the Makefile. Append-ginkgo.label-filter=$(E2E_GINKGO_LABEL_FILTER)to the go test command.- Apply labels
"Platform:AWS","Platform:GCP"and"Platform:IBM"to cases description accordingly.- Set the default value of
E2E_GINKGO_LABEL_FILTERin the Makefile to"Platform: isSubsetOf {AWS}".- It would then select and run both non-platform-specific and AWS-only cases.
Other changes:
- Remove some hardcoded skipping logic.
- Regroup ACME dns-01 cases by their provider platform. Each dns-01 provider now will have a dedicated
Context().I've tested on the GCP cluster. It shows: (skipped 3 cases labeled
"Platform:AWS", 1 labeled"Platform:IBM")go test \ -timeout 1h \ -count 1 \ -v \ -p 1 \ -tags e2e \ -run "" \ ./test/e2e \ -ginkgo.label-filter="Platform: isSubsetOf {GCP}" ... Ran 17 of 21 Specs in 524.481 seconds SUCCESS! -- 17 Passed | 0 Failed | 0 Pending | 4 Skipped --- PASS: TestAll (524.49s) PASS ok github.com/openshift/cert-manager-operator/test/e2e 846.604s
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.
@lunarwhite: all tests passed!
Full PR test history. Your PR dashboard.
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.
@lunarwhite: new pull request created: #251
In response to this:
/cherry-pick cert-manager-1.15
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.
