epiphany
epiphany copied to clipboard
[FEATURE REQUEST] [epicli test] Print total number of failed tests
Is your feature request related to a problem? Please describe.
epicli test
may produce a big number of xml files (in spec_tests
sub-directory) and there is no information about total failed tests.
One has to check all xml files manually, for example using grep:
grep -Por 'failures="\d+"' /workspaces/epiphany/clusters/build/alma-test-big/spec_tests | grep -cv '"0"$'
1
Describe the solution you'd like
Print aggregated test results, for example:
Total tests: 64 [failures: 1, errors: 0, skipped: 0]
Describe alternatives you've considered Configure rake to create single (common) output file for all groups and hosts.
Additional context n/a
DoD checklist
- Changelog
- [ ] updated
- [ ] not needed
- COMPONENTS.md
- [ ] updated
- [ ] not needed
- Schema
- [ ] updated
- [ ] not needed
- Backport tasks
- [ ] created
- [ ] not needed
- Documentation
- [ ] added
- [ ] updated
- [ ] not needed
- [ ] Feature has automated tests
- [ ] Automated tests passed (QA pipelines)
- [ ] apply
- [ ] upgrade
- [ ] backup/restore
- [ ] Idempotency tested
- [ ] All conversations in PR resolved
- [ ] Solution meets requirements and is done according to design doc
- [ ] Usage compliant with license
Would be nice to consider another thing related to test results:
- when running tests against the cluster that has configured components that use role mapping for
filebeat
underconfiguration/feature-mapping
, but doesn't includelogging
component, it will generate empty spec_tests result file forfilebeat
- it happens because under
tests/spec/Rakefile
file we have specified to run tests against all groups in inventory, but intests/spec/spec/filebeat/filebeat_spec.rb
we are skipping tests iflogging
component is not enabled - empty files are read by Azure to publish test results which lead to warnings about tests for filebeat
Thanks to @przemyslavic for explanation.
If you think that it requires separate ticket, please comment.