libiff icon indicating copy to clipboard operation
libiff copied to clipboard

Test suite contains inter-test dependencies

Open enter-github-username opened this issue 1 year ago • 0 comments

Several tests are dependent on the output (several *.TEST files) of other tests:

readnestedform and searchforms-nestedform are dependent on writenestedform validcat.sh and readcat are dependent on writecat searchforms-form, lookupproperty-simple, validform.sh, readform and searchforms-cat are dependent on writeform ppextension-c, checkextension and readextension are dependent on writeextension lookupproperty-prop and validlist.sh are dependent on writelist readform-pad is dependent on writeform-pad

Note: Running the test suite once using make check prevents this behaivior from manifesting, as the *.TEST files are not deleted by default after running the test suite.

This is not ideal, as it prevents the parallel execution of the testsuite with e.g.make -j16 check on a freshly cloned repository - as some of the dependent tests are executed before their dependencies, the test files are not yet written to the disk, causing those tests to fail.

Explicit execution of those tests, with e.g. env TESTS="readform" make -e check, which only executes the test readform also fails.

A possible solution may be to specify those inter-test dependencies, like shown in the GNU automake documentation[1] or using a normal make dependency. However, this might require quite a substantial rewrite of the test suite.

Another solution may be to generate and add the required *.TEST files to the git repo.

[1] https://www.gnu.org/software/automake/manual/html_node/Parallel-Test-Harness.html

enter-github-username avatar Feb 19 '23 09:02 enter-github-username