lighthouse
lighthouse copied to clipboard
tests(smoke): add an option and cli flag to control test exclusion
Addresses #14127.
This implements an exclude list to the smoke test definition that should help to exclude a test from a specific runner or all runners.
Example:
export default {
id: 'foo',
expectations,
config,
+ exclude: ['devtools', 'cli'],
};
A test can be excluded from all runners by specifying exclude: ['*'].
In order to temporarily ignore an exclusion for debugging, Smoke CLI now features a boolean flag --ignore-exclusions.
@connorjclark I chose to retain the list as a fixed structure instead of exclude being a boolean.
added some thoughts to #14127 about the approach
Closing this PR; will raise a new one that addresses the feedback.