irrlicht
irrlicht copied to clipboard
minetest
Unit test restoration
I've started adding unit tests back to IrrlichtMt, leaving out all the stuff that tests code we removed. The tests are integrated with CMake and ctest. There was a request to enable test coverage metrics, but I'm not sure that's useful for our purposes, can be discussed. More importantly, how many tests do we want to restore, and if not all of them which parts of the code do we want to keep tests for?
I have about 30 unit tests, but I want to review each test before submitting it, and start by submitting a couple of the most useful tests. The matrix operations test has already indicated matrix errors on OpenBSD, that one is definitely staying in.
When I submit a PR, which CI builds should I enable unit testing on? Relating to this, I need to decide whether to keep the fastAtof unit test, which tests Irrlicht's custom "fast" atof implementation. The test is disabled on Debug mode because it always fails the performance check, and it also failed the performance check on my OpenBSD workstation in Release mode. On one hand it seems wise to test this, on the other hand we aren't likely to mess with that code and the test is going to randomly raise a ruckus because their "fast" implementation is too slow.
When I submit a PR, which CI builds should I enable unit testing on?
Where it can run natively: so Linux and macOS.
I need to decide whether to keep the fastAtof unit test,
We don't need tests which fail but indicate no bug, you can throw it away.
I want to proceed with this, but the PR is blocked by an error on MacOSX that I can't debug, and I'm more aware now that adding all these tests might not be useful. What do you think about just adding the ctest part to provide a hook for regression tests etc., possibly putting the Examples test in there?
PR is working now, and I'm looking into better test runners (should probably be a separate PR) I found doctest, which is a very (and I mean very) lightweight single-header library. It's extremely easy to include in the build, and would make the entire test suite much cleaner. Thoughts?