smalltalkCI icon indicating copy to clipboard operation
smalltalkCI copied to clipboard

Write more verbose/non-deferred logs about test execution to stdout

Open LinqLover opened this issue 3 years ago • 0 comments

As opposed to other testing frameworks such as Python's unittest (with the command line flag -v), smalltalkCI only prints all test results to stdout after the last test has ended. However, while this results in a clean and beautiful output log, it makes it very hard for me to associate any Transcript outputs raised during the tests' execution to a particular test.

Here is an example: image In this job, some tests trigger a transcript output "Nothing to hide"; however, I do not know which concrete tests are that, and the only option I have today is to count the number of single .s/Es that note the execution of each test and then to reconstruct manually which test belongs to the latest ./E before the transcript output.

Tasks like this would be much easier if smalltalkCI could be more verbose and print a comprehensible log like the following for every test:

 Testing project...
 
 Running suite "Squeak64-5.3 on Travis CI (188.2)" with 264 tests.
 
+Running SBEBasicClassesTestCase >> #test101 ... PASSED.
+Running SBEBasicClassesTestCase >> #test102 ... PASSED.
 Nothing to hide
+Running SBEBasicClassesTestCase >> #test103 ... ERRORED.
+Running SBEBasicClassesTestCase >> #test104 ... PASSED.
 # ...
 Finished running suite: Squeak64-5.3 on Travis CI (188.2)
 
 (258 tests passed)
 # ...

Note: The proposal is to add these verbose logs in the test_logs output group. However, the summary of passing and failing tests does not need to be touched, it looks fine. :-)

I'm reporting this observation after I have found myself multiple times counting all the dots and Es manually. Thus this feature request. :-)

LinqLover avatar Oct 14 '20 23:10 LinqLover