pi_piper
pi_piper copied to clipboard
jwhitehorn
Tests using actual devices
I'm personally quite skeptical of using stub drivers for testing, it might not capture real device behaviour and things might go wrong in future releases.
To amend this, how about having a set of tests that will run only if the test suite is being run on a raspberry pi and will actually test the underlying hardware? These tests can be exactly the same as the current ones, only using real device drivers, and they will run only a raspberry pi, nowhere else.
We can also make it mandatory for future contributors to write two sets of tests - one using stub drivers and the other using real drivers. Or maybe both of these can be unified by cleverly using the before block.
Thoughts?
What you are talking about is integration or full stacks tests, which are very very different than the unit tests that currently exist. I'd be very wary or leveraging an entire testing framework on integration tests as they tend to be very slow and very brittle and will fail even when your code is perfectly fine. Unit tests test your code to make sure it is working how you intended with your assumptions about the outside world. As a side affect unit tests tend to be a good way to document your assumptions about the interfaces you are working with. Integration tests test that you are integrated correctly with the outside world. My preference is to write lots and lots of unit tests that run quickly in memory and test all the various different pathways of your own code and write only a few integration tests, that you know make sure the outside world exists.
I agree with @bguest -- I had to maintain a medium-sized suite of integration tests at my last job and it was... well, troublesome. However, having some integration tests is definitely better than nothing.
I think a compromise can be reached between maintaining integration tests and leaving testing to a purely simulated environment (in this case the driver).
Something with which the actual working of the BCM 2385 driver with the processor pins will be tested.