[Feature] Add remediation details for tests that fail or skip during cnf-testsuite runs and output on the command line.
Describe the solution you'd like There is a mix of remediation steps and details for some tests but not all tests. We should add remediation steps for tests that are not so straight forward with a brief summary and possibly a link one can follow to view how to fix when a test fails or is skipped, etc.
**How will this be tested? aka Acceptance Criteria (optional) ** When tests fail or skipped, the testsuite should output remediation steps or link to documentation that helps users find the solutions to pass the test if desired.
Once this issue is addressed how will the fix be verified?
The output will display remediation steps or links when tests fail or are skipped.
Example of missing remediations:
As a user, when the livenessProbe test fails, I would expect to see remediation suggestions
Here is what I currently see: ✖️ 🏆 FAILED: No livenessProbe found ⎈🧫
As a user, when the readinessProbe test fails, I would expect to see remediation suggestions
Here is what I currently see: ✖️ 🏆 FAILED: No readinessProbe found ⎈🧫
@HashNuke
Want to work on this one together? We can identify all the tests missing remediation, split them up between us to write up the remediation steps or links for output and then update the code.
@agentpoyo Sure.
I identified the tests that need work. Attaching a list. I'll share the sheet with you too.
- I am working on changes for some of these tests. I'll add a column for work completion status.
- I'll post back on how we can collaborate. I think I definitely need help setting up failure scenarios (CNFs) for each of these to be able to test. That's where most of time is being spent too because some of our automated spec tests for certain tests do not have a failure scenario test.
Still kinda relevant, but remediation can now be easily found in documentation. Not sure if this issue can be closed or should stay there as a reminder to update outputs for some of the tests.