legion
legion copied to clipboard
slow but correct 1000-job test case causes failure
Hi,
The test in https://github.com/fukamachi/legion/blob/master/t/legion.lisp#L135 hard-codes a 1-1.3× duration as "test success", and anything slower as "test failure".
On my machine, a macbook pro 2015, it can easily reach 10x:
process 1000 jobs
× Expected it to end in 5.0 - 6.5 seconds, but took 44.761331d0
NIL is expected to be T
Should this actually be considered a failing test, though? I don't mind that it's slow, as long as it works. Where does the 1.3x come from? Shouldn't this be a benchmark rather than a unit test?
I guess my worry is that this makes the correctness of the test dependent on the machine running the test, which doesn't seem ideal.