arduino_ci
arduino_ci copied to clipboard
Arduino-CI
Prevent infinite loops with an ability to disable "instantaneous" (0 clock advancement) builtin hardware functions
Issue Summary
In a unit test I have a loop that uses micros() to do a loop for a certain time. As micros() always returns zero the loop will never end as intended.
One should also check millis() ?
Part of this is covered by #145 -- the need to have the mocking framework respond in a hardware-like way (time passing, etc). It's a messy problem at any scale, because we'll be working around the fact that Arduino has never really contended with the idea of dependency injection or testability; as a consequence, libraries may contain a lot of loops that assume (e.g.) a clock that increments all by itself. Which is a reasonable assumption!
The other part of this is to be able to set a timeout for an individual unit test (which should have a sensible default like "60 seconds"), so that the test runner itself can't hang. I'm going to repurpose this issue for that timeout.
#136 has more examples of the necessary behavior, but as this is a more general request I'm wondering if I could get away with a global "API calls cause this much delay" value. That would eliminate the possibility of infinite loops based on time (although not on something like waiting for an input signal). I'll keep this open to track the simplest option, which would be a long of how much to delay every time any one of the functions is called.
Idea also send in email to Ian
think the micros might be implemented simple and quite realistic.
At start of a unit test a global var STARTTIME is set to the system time in some units
micros() { return (system time - STARTTIME) * TIME2US * TIMESPEED; }
millis() { return (system time - STARTTIME) * TIME2MS * TIMESPEED ; }
TIME2US is a constant depending on the accuracy of system time
TIMESPEED Is a positive number that the user can set, (default == 1) to simulate time faster than normal e.g. TIMESPEED = 9.1, will make the clock tick 9.1x faster.
The problem here is the need to make the mocked hardware perfectly deterministic so that you can properly test that the libraries are doing proper calculation.
For an oversimplified example, if you wanted to measure the slope of an analog input signal you'd want to measure change in voltage divided by change in time. To expect an exact value in the unit test, you need to be able to control the inputs -- including the clock -- with that same exactness.
This is not much of an issue unless you are testing a library that (1) uses while loops in functions and (2) uses hardware state as exit criteria for those loops. Would I be mistaken in thinking that using while loops in that way is limited to cases where the underlying hardware is really what's being tested, and not so much the software?
In my ACS712 lib I have exact such a case.
*ACS712.ma_AC() *measures the AC current by measure the voltage as often as possible to determine the peak to peak voltage. and then calculate the amps from that. So I have to measure in a loop that takes a full sine at 50Hz so at least 20 milliseconds. Using a counter is not possible as the performance of the ADC is board dependent.
There are also functions that have a time based handshake e.g. trigger a sensor to read, wait some time and read the value. Think DS18B20, it blocks some time depending on the resolution set.
These functions are hard for unit tests and unless one can inject either predetermined values or connect to a superb simulator these are hard to do. That's why I test my HW dependent libraries with a real HW, these tests form the base for the examples. So in short, I do not expect unit tests to tackle such "difficult" tests.
What I could do however in my library is e.g. split the math part from the sensor reading part. That might make the math part unit testable.
food for thought...
unless one can inject either predetermined values
Have you looked at the GODMODE documentation or examples at all? Injecting values to be read from digital or analog pins is one of the first features I added: https://github.com/Arduino-CI/arduino_ci/blob/master/REFERENCE.md#pin-futures
Yes, seen that However as micros() stays at 0 filing a future array will not solve the issue. I could patch the library code and count the number of samples (expect 200-220 needed ) and let the loop stop when i have enough samples. But then it becomes non production code and makes it no sense to test.
To be continued
Yes... you're right. What would your ideal pattern be for orchestrating a set of state changes for your library code to "discover" during a unit test? I've struggled with this question a fair amount trying to balance "ergonomics" with expressiveness.