Add guide for running tests

[fosslinux]

I am packaging bibletime for Void Linux. One of the steps is trying to run the tests, but I am constantly running into errors, and a whole lot of things have had to be done that are undocumented.

It would be very helpful if how to run the tests was documented, especially so packagers can programmatically ensure that the package is working, at least to some extent.

Two examples of such issues are:

1. Having to run:

export QT_QPA_PLATFORM=offscreen
mkdir -p share/bibletime
cd share/bibletime
mkdir -p icons javascript license pics locale docs/handbook/en docs/howto/en display-templates

To get the tests to run in a command line environment.

2. This error:

FAIL!  : test_CSwordBackend::booknameLanguage(L2) Compared values are not the same
   Actual   (QString(vKey->getBookName())): "John"
   Expected (bookName)                    : "Johannes"
   Loc: [/builddir/bibletime-2.11.2/tests/backend/managers/cswordbackend/test_cswordbackend.cpp(236)]
PASS   : test_CSwordBackend::booknameLanguage(L3)
FAIL!  : test_CSwordBackend::booknameLanguage(L4) Compared values are not the same
   Actual   (QString(vKey->getBookName())): "Genesis"
   Expected (bookName)                    : "1. Mose"
   Loc: [/builddir/bibletime-2.11.2/tests/backend/managers/cswordbackend/test_cswordbackend.cpp(236)]

Both from the latest stable.

[jaakristioja]

I don't know how you ran them, but using ctest seems to work properly for me, with all 5 tests passing.

However, I would currently discourage you to run the tests at all. Most importantly because these might mess up your ~/.sword directory and likely require internet access. There are currently no plans to "fix" them. In my opinion most parts of neither BibleTime or the underlying Sword library are very "testable" right now.

[fosslinux]

Oh, I was running them through make test. I'll try ctest, directly. I will take your advice into consideration, thanks!

[jaakristioja]

FAIL!  : test_CSwordBackend::booknameLanguage(L2) Compared values are not the same
   Actual   (QString(vKey->getBookName())): "John"
   Expected (bookName)                    : "Johannes"
   Loc: [/builddir/bibletime-2.11.2/tests/backend/managers/cswordbackend/test_cswordbackend.cpp(236)]
PASS   : test_CSwordBackend::booknameLanguage(L3)
FAIL!  : test_CSwordBackend::booknameLanguage(L4) Compared values are not the same
   Actual   (QString(vKey->getBookName())): "Genesis"
   Expected (bookName)                    : "1. Mose"
   Loc: [/builddir/bibletime-2.11.2/tests/backend/managers/cswordbackend/test_cswordbackend.cpp(236)]

However, this seems to be like a locale issue.

Testing in BibleTime has not received enough attention and is of poor quality. I suggest packagers not run them at all, as even developers usually don't regularly run these.