Feature Request: Prioritise spec files

[caalberts]

Hi, I'd like to know if you'd be open to adding a feature that allows users to prioritise certain spec files.

This can be used to run test files for classes or modules that have recently changed first. so any failure in these can be reported early, before continuing with the rest of the suite which may be less likely to fail.

It looks like it is quite straight forward to add this by prepending the prioritised tests in the allocator node_tests.

As far as I know, rspec and cucumber would honor the order of the files that are passed in, unless specifically overridden.

[ArturT]

Hi @caalberts

This can be used to run test files for classes or modules that have recently changed first.

How would you detect what tests should be run? Maybe by using some kind of code coverage tool?

Would some other tool provide the list of tests that should be run first or Gitlab already does that?

knapsack could take a list of tests in ENV and run it first.

Something like that would be already possible in knapsack_pro:

You could do it in 2 steps:

Step 1. Run c tests for classes or modules that have recently changed.

export KNAPSACK_PRO_TEST_FILE_LIST=spec/features/dashboard_spec.rb,spec/models/user.rb
export KNAPSACK_PRO_TEST_SUITE_TOKEN_RSPEC=token-for-priorities-tests
bundle exec rake knapsack_pro:queue:rspec

https://github.com/KnapsackPro/knapsack_pro-ruby#how-to-run-a-specific-list-of-test-files-or-only-some-tests-from-test-file

Step 2. Run all tests except tests executed in step 1.

export KNAPSACK_PRO_TEST_FILE_EXCLUDE_PATTERN="{spec/features/dashboard_spec.rb,spec/models/user.rb}"
export KNAPSACK_PRO_TEST_SUITE_TOKEN_RSPEC=token-for-other-tests
bundle exec rake knapsack_pro:queue:rspec

https://github.com/KnapsackPro/knapsack_pro-ruby#how-to-exclude-tests-from-running-them