lighthouse icon indicating copy to clipboard operation
lighthouse copied to clipboard

tests(smoke): add an option and cli flag to control test exclusion

Open alexnj opened this issue 3 years ago • 1 comments

Addresses #14127.

This implements an exclude list to the smoke test definition that should help to exclude a test from a specific runner or all runners.

Example:

 export default {
   id: 'foo',
   expectations,
   config,
+  exclude: ['devtools', 'cli'],
 };

A test can be excluded from all runners by specifying exclude: ['*'].

In order to temporarily ignore an exclusion for debugging, Smoke CLI now features a boolean flag --ignore-exclusions.

@connorjclark I chose to retain the list as a fixed structure instead of exclude being a boolean.

alexnj avatar Jun 16 '22 10:06 alexnj

added some thoughts to #14127 about the approach

brendankenny avatar Jun 17 '22 17:06 brendankenny

Closing this PR; will raise a new one that addresses the feedback.

alexnj avatar Aug 24 '22 01:08 alexnj