nextest icon indicating copy to clipboard operation
nextest copied to clipboard

Run fast/slow tests first

Open fenollp opened this issue 1 year ago • 8 comments

Hello! I'd like the queue of tests to run to be ordered by the time each took last time nextest was run. It's a simple and weak heuristic with little impact on the call.

Maybe this can be opt-in with a flag a la --ordering=fast-first

fenollp avatar Jul 12 '23 12:07 fenollp

Hi there --

This would be awesome to have, the only issue is that we don't record previous test runs at the moment. There's some design work that needs to happen to support that.

sunshowers avatar Jul 12 '23 16:07 sunshowers

#958 is about running the slowest tests first, which is something that would also be useful.

sunshowers avatar Sep 14 '23 17:09 sunshowers

In the process of implementing this, would it be possible to publicly expose the ability to dump the execution statistics that nextest displays during execution in a machine-friendly format ? I'd like to have that data sorted by decreasing time as a kind of execution profile to know which tests I should optimize first.

HadrienG2 avatar Oct 26 '23 08:10 HadrienG2

@HadrienG2 I think that would be part of #20 if I'm understanding you correctly.

sunshowers avatar Oct 26 '23 19:10 sunshowers

Probably, I was just thinking that implementing this feature requires persistifying exactly the data I want ;)

HadrienG2 avatar Oct 26 '23 21:10 HadrienG2

Since https://github.com/nextest-rs/nextest/issues/1152 has started progressing a little, can we get some guidence as for what's the next blocker for this? :no_mouth: As hinted to by the referring PR above, we'd be interested in running the tests in decreasing order of duration. It significantly improves multi-core utilization, therefore decreases overall testing time.

torokati44 avatar Apr 18 '24 15:04 torokati44

As a stop-gap, would it be possible to run tests in random order?

matthiasgoergens avatar May 21 '24 05:05 matthiasgoergens