moodle-local_ci icon indicating copy to clipboard operation
moodle-local_ci copied to clipboard

Refactor the unit tests (from whole-file fixture assertion) to better check only the important bits

Open stronk7 opened this issue 9 months ago • 0 comments

Right now, for a lot of tests (specially CiBoT ones, but also others), we are following a "whole-xml" fixture strategy, so we compare, char by char, that the results are 100% the same than a given fixture file.

While that works, it's not manageable at all, because every single change anywhere leads to N (dozens) of failures in tests that are completely unrelated.

To say it with other words, with actual information, one change in the CiBoT reports, adding a new section, was straightforward and implemented and tested in < 2h. But, since then, I'm spending > 8h manually amending lots of "whole-xml" fixture files to get the tests passing again.

So, we should be able to move to some alternative approach where, in each test, we only look for the relevant pieces of information. It can be substring matching, or regex matching or whatever, but we should reduce the "thole-xml" tests to the minimum.

Doing this will save us, long term, lots and lots of hours of boring, useless and repetitive cycles. Let's make each test to verify, exclusively, it's own stuff, not the "whole-thing".

Ciao :-)

stronk7 avatar May 16 '24 07:05 stronk7