ci: Aggregate otter tests results for testContainer
Description
This pull request introduces improvements to how output directories and test artifacts are managed and uploaded for Otter integration and container tests. The changes make the test environments more configurable, ensure logs and reports are better organized per test, and update CI workflows to use the new artifact locations.
Test Environment and Output Directory Enhancements:
- Added an
outputDirectory()method to theTestEnvironmentinterface and implemented it inContainerTestEnvironment, allowing tests to specify and retrieve their output directories for artifacts. [1] [2] - Refactored
ContainerTestEnvironmentto accept a custom output directory, defaulting tobuild/aggregateTestContainer, and updated constructors accordingly. [1] [2] [3] - Introduced a new abstract
BaseIntegrationTestclass that sets up a unique output directory for each test run and provides factory methods for creatingTurtleTestEnvironmentandContainerTestEnvironmentinstances with the correct output paths.
JUnit Extension and Lifecycle Improvements:
- Updated the
OtterTestExtensionJUnit extension to set up a unique output directory for each test before execution and store it in the extension context, supporting both "turtle" and "container" environments via a factory function. [1] [2] [3] - Added a new
Lifecyclerecord to manage the lifecycle of aTestEnvironmentwithin the JUnit extension context.
CI/CD Workflow Updates:
- Changed artifact upload paths in
.github/workflows/node-zxc-compile-application-code.yamland.github/workflows/zxc-execute-otter-tests.yamlto use the new aggregated directories, improving log organization and retention. [1] [2] [3] - Updated workflow manifest and test result file references to match new output locations and naming conventions. [1] [2] [3] [4]
These changes make test output management more robust and flexible, simplify artifact uploads, and ensure that logs and reports are consistently organized per test run.
Related Issue(s)
Closes #21957
:white_check_mark: Snyk checks have passed. No issues have been found so far.
| Status | Scanner | Total (0) | ||||
|---|---|---|---|---|---|---|
| :white_check_mark: | Open Source Security | 0 | 0 | 0 | 0 | 0 issues |
:computer: Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.
Codecov Report
:white_check_mark: All modified and coverable lines are covered by tests.
@@ Coverage Diff @@
## main #21979 +/- ##
============================================
+ Coverage 70.89% 70.94% +0.04%
Complexity 24432 24432
============================================
Files 2668 2668
Lines 104410 104336 -74
Branches 10966 10955 -11
============================================
- Hits 74020 74019 -1
+ Misses 26344 26274 -70
+ Partials 4046 4043 -3
see 20 files with indirect coverage changes
:rocket: New features to boost your workflow:
- :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
Coverage summary from Codacy
See diff coverage on Codacy
| Coverage variation | Diff coverage |
|---|---|
| :white_check_mark: +0.05% (target: -1.00%) | :white_check_mark: ∅ |
Coverage variation details
| Coverable lines | Covered lines | Coverage | |
|---|---|---|---|
| Common ancestor commit (88c39cfc54018f6cf1afe7e052a7bf92f423fe15) | 104315 | 78021 | 74.79% |
| Head commit (db4b1871f5304512a8e8b7ae7688701ae89c5962) | 104241 (-74) | 78017 (-4) | 74.84% (+0.05%) |
Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: <coverage of head commit> - <coverage of common ancestor commit>
Diff coverage details
| Coverable lines | Covered lines | Diff coverage | |
|---|---|---|---|
| Pull request (#21979) | 0 | 0 | ∅ (not applicable) |
Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: <covered lines added or modified>/<coverable lines added or modified> * 100%
See your quality gate settings Change summary preferences
Local testing passes:
XTS Dry Run with Otter Tests After Rebase.