Figure out a way to test the frontends [$40]

[quarnster]

We really, really need to figure out how to write tests for the frontends. #373 related. There's probably much overlap here between frontends also so perhaps we can invent a way to have a shared test suite.

--- There is a **[$40 open bounty](https://www.bountysource.com/issues/4376331-figure-out-a-way-to-test-the-frontends?utm_campaign=plugin&utm_content=tracker%2F282001&utm_medium=issues&utm_source=github)** on this issue. Add to the bounty at [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F282001&utm_medium=issues&utm_source=github).

[erbridge]

Maybe useful for qml: http://qt-project.org/doc/qt-5/qttest-qmlmodule.html

[kennylevinsen]

I am currently doing some restructuring of the termbox frontend, mainly focused on the loop function having far too much of the logic implemented.

The result should be closer to be unittestable (things like testing how it handles resize, how it handles input, etc. without having to let loop run), as well as making it easier to follow. The QML frontend could use some of the same love.

[njskalski]

One of the ways to test QT based UI is via IAccessible interface, which is available pretty much out-of-box for QT. It might be a good idea to support accessibility features in text mode as well. This would give us at least partially unified test experience.

[techtonik]

A little bit of image recognition may help - Sikuli X http://sikulix-2014.readthedocs.org/en/latest/basicinfo.html#sikulix-can-be-used-for-visual-testing

[quarnster]

Sikuli X seems a bit overkill, but capturing and comparing "gold images" could indeed be a good way to test some features.

[erbridge]

From #373:

@pjoe:

QML has a testing framework that might be suited for this: http://doc.qt.io/qt-5/qtquick-qtquicktest.html

I have experience using this in my day job, feel free to contact me on IRC

[quarnster]

Where possible, I'd like to have the tests as un-frontend-specific as possible. I.e. we'd find a way to write tests that abuse the Frontend-interface (which might need to be extended) without knowing which frontend it's actually testing. That way when a new frontend is added it can make use of the full test suite already written. I'd also like the frontend tests to ensure that all frontends behave the same way, and show the same thing, obviously in their own way/style but going from using one frontend to another should be as unsurprising and smooth as possible as an ideal v1.0 goal.