content icon indicating copy to clipboard operation
content copied to clipboard

Templating of test scenarios in the test suite is messy

Open matejak opened this issue 3 years ago • 2 comments

How templated tests are handled:

Issues with this:

  • Templated tests are identified twice - once so scenarios can be collected, and for the second time so that they can get transferred to the testing machine. This increases maintenance costs. A cleaner approach would be to make a list once and then just pass it on in a read-only mode.
  • The function rule_should_be_tested got really complex, it is no longer clear what are the criteria that result in "pass".
  • The preference of rule scenarios over templated ones is achieved by performing file copy in a certain order, the intention to prefer something doesn't have any imprint in the source code.

matejak avatar Jan 12 '22 15:01 matejak

Some of the items will be improved by https://github.com/ComplianceAsCode/content/pull/8751

jan-cerny avatar May 13 '22 07:05 jan-cerny

The templated tests are no longer collected twice since https://github.com/ComplianceAsCode/content/pull/8883

jan-cerny avatar Jul 28 '22 12:07 jan-cerny