hyrax icon indicating copy to clipboard operation
hyrax copied to clipboard

Allow developers to download and run only the tests that failed in CI.

Open tpendragon opened this issue 3 years ago • 1 comments

Descriptive summary

Running the full test suite takes two hours, but CI runs them in ~ 12 minutes, so most developers push their branches to CI to run the whole suite. It would be good if we can download the tests that failed so we can run just those locally.

I think we can do this if the parallel jobs store spec/example.txt, and then combine those files. Then we can download the combination and run rspec spec --only-failures.

Rationale

Let's make it easy to run tests.

tpendragon avatar Mar 11 '22 20:03 tpendragon

After a little research: Each parallel run does indeed generate its own spec/example.txt - they look like this:

example_id                                                                                 | status  | run_time        |
------------------------------------------------------------------------------------------ | ------- | --------------- |
./spec/controllers/hyrax/file_sets_controller_spec.rb[1:1:1:1:1]                           | passed  | 2.26 seconds    |
./spec/controllers/hyrax/file_sets_controller_spec.rb[1:1:2:1]                             | passed  | 2.1 seconds     |
./spec/controllers/hyrax/file_sets_controller_spec.rb[1:1:3:1:1]                           | passed  | 1.94 seconds    |
./spec/controllers/hyrax/file_sets_controller_spec.rb[1:1:3:2:1]                           | passed  | 2.03 seconds    |
./spec/controllers/hyrax/file_sets_controller_spec.rb[1:1:3:3:1]                           | passed  | 2.04 seconds    |
./spec/controllers/hyrax/file_sets_controller_spec.rb[1:1:3:4:1:1:1]                       | passed  | 3.32 seconds    |

tpendragon avatar Mar 11 '22 21:03 tpendragon