FLiT icon indicating copy to clipboard operation
FLiT copied to clipboard

Install FLiT's automated tests

Open mikebentley15 opened this issue 5 years ago • 2 comments

Feature Request

Describe the new feature: We install pretty much everything, except we do not install the tests that test FLiT itself. It may be beneficial to install at least the relevant tests (e.g., do not install the test the checks that we can install FLiT).

Currently, the only automated tests installed are those inside of the python scripts (as doctests). One reason to install the tests is so that the doctests within the python scripts can reuse some of the testing infrastructure used by the FLiT automated tests.

Suggested change:

  • Install relevant tests into $PREFIX/share/flit/tests
  • Add a test_dir variable in flitconfig.py
  • Utilize the test_harness.py inside of doctests in the delivered python scripts

This could make for more rigorous testing by running the automated tests after installation to make sure the installation did not break any functionality.

Alternative approaches: Just install the test_harness.py somewhere to be used by tests and add a variable in flitconfig.py to find it. We don't necessarily need the tests installed. I doubt anyone would even notice that they are even there nor know how to run them. But, if we install test_harness.py somewhere (e.g., $PREFIX/share/flit/test_harness/test_harness.py), then we can comfortably use them from within the doctests. I just worry about having broken doctests after installation.

mikebentley15 avatar Jul 22 '19 16:07 mikebentley15

Specifically, I want to simplify the doctests for flit_bisect.py function run_make().

mikebentley15 avatar Jul 22 '19 16:07 mikebentley15

Also, simplify doctests for extract_make_var() and extract_make_vars() in flitutil.py.

mikebentley15 avatar Jul 22 '19 17:07 mikebentley15