kubeshop
Enhancement to Generate tests CRD file based on directory
Describe the enhancement you'd like to see
The current testkube generate tests-crds works on the reading the file in the directory where are tests stored.
The enhancement should include the following:
- possibility to generate test-suites
- possibility to use additional flags for each test or test-suite (different, not the same for all)
The current implementation takes each file one-by-one, autodetect test type, and creates manifest based on the that including the same flags for every test. However, each test may require different flags, especially when you need to set some arguments for specific test types.
generate can therefore work with definition (JSON, YAML) that will contain information that should be applied to the specified test:
- resource you want to generate (
test,testsuite, or maybe also other) - flags you want to apply for the resource
I believe this would be helpful for not only GitOps.
I know there is a possibility to create you YAML manifest manually and sync it with your environment using GitOps, but the generate functionality helps also to automate adding new tests that are written by someone not aware of testkube or GitOps.
Additional context
Hi there!
For test suites, maybe it would be better to create separate command that would base on prior generated test CRDs?
That way we could introduce actual automated flow, that would allow to create multiple test suites automatically from the repository.
Example proposed syntax, to describe how it could look like:
testkube generate testsuite <name> [--parallel] [-f additional filters] [...crd path patterns]
# Examples
testkube generate testsuite all-suite .
testkube generate testsuite web-suite ./web/*.e2e.yaml
testkube generate testsuite web-cypress-suite -f executor=*cypress* ./web/*.e2e.yaml
