Running `rails test` doesn't automatically load any custom tasks under `lib/tasks`

[onlynone]

Steps to reproduce

1. Create the file lib/tasks/test.rake with the content:

ENV["test.rake"] = "yes"
task :custom_task do
  ENV["custom_task"] = "yes"
end

task :test => [:custom_task]

2. Create a test file at test/unit/rails_test_test.rb with the content:

class RailsTestTest < ActiveSupport::TestCase
  test "test.rake is in the environment" do
    assert_equal('yes', ENV['test.rake'])
  end

  test "custom_task is in the environment" do
    assert_equal('yes', ENV['custom_task'])
  end
end

3. Run bin/rails test test/unit/rails_test_test.rb
4. Run bin/rake test test/unit/rails_test_test.rb

Expected behavior

Both rails test and rake test should succeed.

Actual behavior

Running the tests with bin/rake succeeds with the output:

bin/rake test test/unit/rails_test_test.rb
Running via Spring preloader in process 59092
Run options: --seed 6948

# Running:

..

Finished in 0.000943s, 2120.8907 runs/s, 2120.8907 assertions/s.
2 runs, 2 assertions, 0 failures, 0 errors, 0 skips

While running with bin/rails fails with the output:

Running via Spring preloader in process 58735
Run options: --seed 33633

# Running:

F

Failure:
RailsTestTest#test_custom_task_is_in_the_environment [/Users/steven/projects/rails_test_tasks/test/unit/rails_test_test.rb:7]:
Expected: "yes"
  Actual: nil


rails test test/unit/rails_test_test.rb:6

F

Failure:
RailsTestTest#test_test.rake_is_in_the_environment [/Users/steven/projects/rails_test_tasks/test/unit/rails_test_test.rb:3]:
Expected: "yes"
  Actual: nil


rails test test/unit/rails_test_test.rb:2



Finished in 0.001378s, 1451.3788 runs/s, 1451.3788 assertions/s.
2 runs, 2 assertions, 2 failures, 0 errors, 0 skips

Because the file lib/tasks/test.rake is never loaded.

It seems like rails will only end up loading custom tasks if it can't find the command being run on the command line in its own list of built-in commands. When it can't find the command, it will load the local Rakefile via find_by_namespace("rake").perform(full_namespace, args, config) in Rails::Command.invoke. And the local Rakefile usually has Rails.application.load_tasks and that loads any custom tasks under lib/tasks.

For example, this happens when the ActiveRecord migration exec's out to call: bin/rails db:test:prepare. That task is apparently not a built-in rails task, so the Rakefile and all custom tasks do end up getting loaded for that one.

(As an aside here, I don't fully understand the difference between the built-in rails tasks and the ones that have to be searched for via rake. I also don't understand why the migration code needs to fork a separate process to do db:test:prepare. That task just ends up doing db_namespace["test:load"].invoke. Why couldn't the db:test:prepare tasks simply be invoked the same way in the same process?)

System configuration

Rails version: 6.0.5

Ruby version: 2.6.5

[onlynone]

For some background as to why we'd want custom tasks to be loaded for bin/rails test:

We do some important setup in test/test_helper.rb that we always want to run before any tests. Most of our test files end up loading this file either directly or via other required files. But sometimes that doesn't happen. And if that test file happens to run first during bin/rails test, or it's run directly via bin/rails test test/unit/some_test.rb, then we can have issues.

To solve this, we wanted to create a task that always ran prior to the test task and that task would require test/test_helper.rb. This worked great when we were running things via bin/rake test. But it seems like rails is moving away from invoking tasks with rake: https://github.com/rails/rails/issues/27088#issuecomment-261381489. It looks like rails now supports thor commands... But I couldn't figure out how to make a custom command a prerequisite for test.

[onlynone]

One of the comments on #27088 said:

Right now it is not possible to extend the rails commands, so you are not able to add dependencies to the rails test command, but we are working on this for Rails 5. https://github.com/rails/rails/pull/26414.

However #26414 was merged, but it seems like there's still no way to add dependencies to the rails test command.

And another comment says:

This is also going to change in Rails 5.1. We are not going to pass things to rake anymore.

But it looks like many things still are passed to rake even in rails-7.

[ghiculescu]

Somewhat related: https://github.com/rails/rails/pull/46664

[rails-bot[bot]]

This issue has been automatically marked as stale because it has not been commented on for at least three months. The resources of the Rails team are limited, and so we are asking for your help. If you can still reproduce this error on the 7-0-stable branch or on main, please reply with all of the information you have about it in order to keep the issue open. Thank you for all your contributions.