build
build copied to clipboard
shipwright-io
[FEATURE] Use Ginkgo Labels to Isolate Features in e2e Tests
Is there an existing feature request for this?
- [X] I have searched the existing feature requests
Is your feature request related to a problem or use-case? Please describe.
The current e2e test suite for shipwright/build contains tests for multiple features, however it can only be run as an "all or nothing" approach. We should be able to annotate/label tests to identify the associated feature and run subsets of e2e tests.
Use case: Downstream distributions may not support all the features in the "upstream". The build repo also tests the samples, which are not guaranteed to work on all Kubernetes distributions (OpenShift in particular has skews around security that can make these tests require additional setup).
Describe the solution that you would like.
Ginkgo v2 has "primary" support for labels in spec descriptors [1]. These can be used to narrow tests. We can use labels like [FEATURE: xxx] to narrow features, and others like [CORE] to identify specs that test the most core/stable features.
[1] https://onsi.github.io/ginkgo/#spec-labels
Describe alternatives you have considered.
We could leave descriptions in place and add description-based filtering [1].
[1] https://onsi.github.io/ginkgo/#description-based-filtering
Anything else?
The ginkgo docs recommend using the ginkgo cli to run tests and pass in the right flags to do this kind of filtering.
We might be able to keep the existing go test -based approach by passing in flags via -ginkgo.nnn [1]
[1] https://github.com/onsi/ginkgo/issues/381
From Refinement, @adambkaplan to move this to a Community session, for a group discussion.
Moving this to the "Backlog" since this issue has been carried across multiple release milestones. This issue should be moved to an appropriate release milestone once demonstrated progress has been made (ex: an opened pull request).
@ayushsatyam146 can we pick this item? I have people looking for new items.