libadm icon indicating copy to clipboard operation
libadm copied to clipboard

tests: read test data directly from the source dir

Open tomjnixon opened this issue 3 years ago • 3 comments

These used to be copied to the build dir, but they tended to get out of date, as they were only copied during configuration.

With this, the test data directory is baked into the test executables, so they can be ran outside of ctest easily.

This adds a function to get the full path of a test data file, so if we want to change this again it should be a bit easier.

Fixes #77

tomjnixon avatar Apr 30 '21 16:04 tomjnixon

Codecov Report

Merging #78 (b78a5d9) into master (146670a) will not change coverage. The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##           master      #78   +/-   ##
=======================================
  Coverage   89.51%   89.51%           
=======================================
  Files         115      115           
  Lines        5534     5534           
=======================================
  Hits         4954     4954           
  Misses        580      580           

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 146670a...b78a5d9. Read the comment docs.

codecov-commenter avatar Apr 30 '21 16:04 codecov-commenter

This looks good, but a couple of questions

  • I think the behaviour previously was that with a messed up working dir, all the tests would fail, but now they will pass but you'll get some file output wherever you've run them from? Should we hard code the output dir as well or is that OK?
  • Previously .accepted and .received files were next to each other, which made doing a diff trivial, now you have to know where both live. Should we just output to the source dir and .gitignore the received files? Or is that a terrible idea?

rsjbailey avatar May 05 '21 13:05 rsjbailey

I think the behaviour previously was that with a messed up working dir, all the tests would fail but now they will pass but you'll get some file output wherever you've run them from?

The current implementation will sometimes write .received.xml files to the CWD (for tests which don't read xml files, but do compare them). I think this makes that worse, but it was already a problem.

Should we hard code the output dir as well or is that OK?

Personally I'm happy with this, because git makes it pretty easy to tidy up, and it doesn't generally happen since in most cases people will not run tests manually. I wouldn't be opposed to changing it, though.

Previously .accepted and .received files were next to each other, which made doing a diff trivial, now you have to know where both live. Should we just output to the source dir and .gitignore the received files? Or is that a terrible idea?

I hadn't considered that. Other solutions would be to have FileComparator write the accepted file to the build dir on test failure(though then if you want to edit it, you might edit the wrong one), or to print the both path names on failure (unfortunately it wraps the output of describe()...).

I don't really mind writing them to the source dir though.

tomjnixon avatar May 05 '21 13:05 tomjnixon