Partest should print test number/test file before running the test so that e. g. hangs can be diagnosed immediately

[scabug]

trafficstars

I think it would be better to have the test status at the right, so that the line with the test number and the test file can be printed before the test is executed. When the test is finished, the result is added at the right side.

E. g.

& starting 1347 tests in run
   1 - run/absoverride.scala                             ok
   2 - run/amp.scala                                     ok 
   3 - run/adding-growing-set.scala                      

In the above example, test 3 would be running.

In cases where a test hangs, it is extremely annoying to not immediately see on which test partest is stuck. It would also make test failures more intuitive, because the e. g. stacktrace would come after and not before the test number/file name/test status line.

See https://github.com/scala/scala/pull/2418#issuecomment-16892202

[scabug]

Imported From: https://issues.scala-lang.org/browse/SI-7416?orig=1 Reporter: @soc Affected Versions: 2.11.0-M2

[scabug]

@paulp said: In the example above, I guess you are running with one thread? What's that take, 5-6 hours? Because if you're not running one test at a time, you're not going to find it so easy to produce this output.

[scabug]

@soc said: I had that in mind, but if som-snytt has some Swing UI for it, I think it should at least be possible with that interface. Making it work in the console is most likely a lot more complicated, I agree.

[scabug]

@som-snytt said: It's not so easy.

The other side-effect is that you're waiting for the long-runner to say ok while other tests complete, then it spews all the oks until the next running test. It's more fun to see notifications as they occur (out of submit order).

For this use case, maybe you really want per-test timeout.

As recently noted, it would be nice if it could tell you when space or time requirements for a test appear to deviate. (Even: this test used to complete in five seconds and now it takes twenty. Maybe a Jenkins-like indicator to accompany, if not stormy weather then the rumblings of distant thunder.)

[scabug]

@soc said (edited on Apr 25, 2013 2:44:34 PM UTC):

It's not so easy.

I agree and I wouldn't have created an issue if I hadn't been told to do so. :-)

[scabug]

@som-snytt said: The other advantage to reporting in stable submit order (stable if you sort within the test kind, which partest doesn't do again ATM) is that you can add an option +103 to skip the first 103 tests, or whatever.

I'm doing a test clean-up (strangely interesting exercise), and it would be nice to skip what I now know to be clean.

It's like returning the next day with your tooth brush, only to find that you have to do the urinals again before moving on to the showers.

[som-snytt]

currently

ok    7 - run/Course-2002-02.scala                [duration 5.11s]

Maybe sbt now offers a test progress indicator.

I came looking for a ticket about how a pos test does not show -Vdebug -Vlog output, even under --verbose.