Automatic testing

[cochrane]

I strongly believe unit tests and other automated tests can be very useful, and I think that's true for OpenTomb as well. I'm just not sure how. Here's my general thoughts, just throwing them at the wall and seeing what sticks:

• Unit tests on module layer. There are a lot of things that cannot be usefully tested with unit tests, because they require the full app working together. But there are many parts that can and that have caused issues before (e.g. texture atlas). For those using unit tests could help. We just need to figure out a useful C++ unit test framework. There are millions out there, and I have no idea which one is best (the one I use for my private projects is OS X only; no point even thinking about that one).
• Fully automated game runs. These would be lua scripts that send action messages, for example something like (pseudo-code only):

runToPosition(1024, 1024, 1024)
turnTowards(NORTH)
setKey(ACTION, true)
wait(100)
setKey(ACTION, false)
assertPosition(1024, 1512, 1024)

I did some preliminary work on that one, and it seems like I could get it working eventually, but there are still a number of obstacles to overcome. I imagine this would be most useful in smaller doses, e.g. scripting single, known-to-be-problematic levels or parts of levels. Note that this would not just send keys, but also read back current positions and so on to deal with variations in behavior. Sending just key presses with delays is guaranteed to lead to madness.

Could either of this work and be useful? Does anyone have other suggestions? The testing approach right now is clearly not sufficient (whenever I do something I end up breaking TR3, because I rarely test on that one, for example), and I think this might help.

[stohrendorf]

I'd prefer Boost Test, but that's mainly because I love Boost, and I don't know other test systems. A benefit of Boost Test would be that it introduces Boost in general, which (despite the additional dependency) I consider a possible improvement, because many new c++ standards/extensions have their origin in Boost anyway.

[vvs-]

Oh yes! That would be definitely most useful.

I just suggest that it don't use the realtime engine as it is now. It's non-deterministic. There should be a simulation framework. At the bare minimum it should simulate time intervals.