Snaipe
Provide a minimal no-dependency server/client system for embedded devices
Embedded systems are not guaranteed to provide a POSIX environment or have the hardware requirements to even run the full runner.
It could be a nice addition for embedded developers to compile the hardware tests in "client" mode. The test runner would then run on some kind of "server" mode that listens for any data stream from the client.
Food for thought:
- Should the server start the client test itself, by asking a user-defined function that handles the logic of launching the test, or should it just wait for the client to connect by itself ? (maybe add both options and let the user decide)
- Should the server try to upload the client source or client binary to the device, using an user defined function, or should it assume that the device has the client and is already ready for testing?
- How should parameterized tests or theories be handled on hardware tests ?
One thing that is clear is that most of the handling logic will have to be provided by the user himself. This includes streaming data between client and server, compiling the client and programming the chip with it, running the client, ...
My dream would be to have a test framework that allows for CI and running of test suites on real world ARM Cortex-M1/M3/M4 devices. CI on real hardware after every pull request, merge, etc.
A CI tool would cross compile the code and then run openocd or similar to flash the device with the latest firmware build with some interface to communicate with the "server" runner. The device under test should reset after programming and be waiting for communications via a well defined communications stream. As long as a bidirectional stream is possible, any transport could be implemented such as BLE, UART, ARM semihosting, TCP/IP.
The server process on the host PC would then start the test on the target embedded processor. This may require a short arbitrary message is sent to the embedded device over a UART (or similar) and then enter the client-server communications interface.
Additionally, embeddeding the tests completely in the firmware would allow a minimal version of criterion to serve as a POST for the device. In this manner, invoking the test suite by a function call would be great idea.
An interesting test framework that targets embedded devices is https://cmocka.org/
