fiasco icon indicating copy to clipboard operation
fiasco copied to clipboard

Fix the report of test execution times

Open hineios opened this issue 5 years ago • 4 comments

Test execution reports should report the real time spent testing instead of the internal run time.

hineios avatar May 14 '20 11:05 hineios

As requested, a separate pull request for the time reporting correction. :)

hineios avatar May 15 '20 09:05 hineios

I'd like to bring @luismbo to discuss this one. Stefil, Fiasco's predecessor used get-internal-run-time, and that's why it's here. But indeed prove and lift use get-internal-real-time. fiveam and unit-test use nothing. I don't know what all the other million CL test frameworks use.

I can see arguments for and against each one. "real time" is a measure of how long we have to wait. But "run time" is a measure of how much time the processor(s) have spent with our tests. If we have a hugely overloaded system, the second should stay more or less the same, right?

Another idea is: show both. That is what time does anyway.

joaotavora avatar May 15 '20 11:05 joaotavora

runtime is annoying for tests that launch external processes, talk to slow databases, etc. We could not show anything, that'd work too. :-)

luismbo avatar May 15 '20 23:05 luismbo

OK I'm sold. I'll merge this tomorrow, once I fix all the indentation in a separate commit.

joaotavora avatar May 15 '20 23:05 joaotavora