system-tests
system-tests copied to clipboard
Manifest references + Node semver migration
trafficstars
Motivation
Introduces a special optional field refs in manifests, it should hold a list of reference definitions, to be used elsewhere in the manifests.
Example:
refs:
- &version_A '>=5.6.0 || 4.7.0'
- &version_B v2.3.0
tests/:
file.py:
ClassA: *version_A
ClassB: *version_B
It synergizes particularly well with SemVer support, to reduce noies and potential errors.
Changes
- Add documentation on the feature and recommendations on using it.
- Change all the Node.js versions with their full SemVer descriptors and use a ref table for it.
- Add a special case in manifest parsing to not consider the
refsfield as a test file. - Update the JSON Schema for the manifests.
Workflow
- ⚠️ Create your PR as draft ⚠️
- Work on you PR until the CI passes (if something not related to your task is failing, you can ignore it)
- Mark it as ready for review
- Test logic is modified? -> Get a review from RFC owner. We're working on refining the
codeownersfile quickly. - Framework is modified, or non obvious usage of it -> get a review from R&P team
- Test logic is modified? -> Get a review from RFC owner. We're working on refining the
:rocket: Once your PR is reviewed, you can merge it!
Reviewer checklist
- [ ] Relevant label (
run-parametric-scenario,run-profiling-scenario...) are presents - [ ] If PR title starts with
[<language>], double-check that only<language>is impacted by the change - [ ] No system-tests internal is modified. Otherwise, I have the approval from R&P team
- [ ] CI is green, or failing jobs are not related to this change (and you are 100% sure about this statement)
- [ ] A docker base image is modified?
- [ ] the relevant
build-XXX-imagelabel is present
- [ ] the relevant
- [ ] A scenario is added (or removed)?
- [ ] Get a review from R&P team
Job is failing because the system-tests transforms the valid semver version provided by the Node library 6.0.0-pre into 6.0.0pre which is not a valid string
ok it's fixed now :+1: