vunnel icon indicating copy to clipboard operation
vunnel copied to clipboard

Test: add quality gate for percentage of namespaces exercised by at least one test image

Open spiffcs opened this issue 2 years ago • 0 comments

What would you like to be added: During the quality gate we want to check if results are being tested from > 70% of namespaces.

Example: coverage of namespaces sampled is at 71%, a user adds a new provider that adds a new namespace, coverage would drop < 70% causing the gate to fail since the new namespace is not yet exercised by at least one test image.

Why is this needed: To catch if there are failures using our test data against current/new providers before running a vunnel release

Additional context: The tools team is able to point to a Post-Mortem regarding missing debian data that resulted in this issue being filed

spiffcs avatar Sep 29 '23 19:09 spiffcs