SimpleCov rspec output breaks test detection

[maxlorenz]

Your environment

• vscode-ruby-test-adapter version: 0.7.1
• Ruby version: 2.5.5
• VS Code version: 1.46.0-insider
• Operating System: Windows 10
• RSpec version: 3.8

Expected behavior

With SimpleCov enabled, the extension should detect the specs

Actual behavior

The spec detection step fails silently. SimpleCov adds a few lines after the dry-run output, which may cause this behavior. Disabling the coverage report fixes it at least

[connorshea]

Could you give more information on how exactly SimpleCov is configured and what it's printing out? I've got two projects with SimpleCov enabled but neither has this problem.

[pbnj]

I'm running into a similar issue.

The Ruby Test Explorer Log shows the following error:

[2020-12-16 23:43:45.196] [INFO] Running dry-run of RSpec test suite with the following command: bundle exec rspec --pattern './spec/**/*_test.rb,./spec/**/test_*.rb,./spec/**/*_spec.rb' --require /Users/pbenjamin/.vscode/extensions/connorshea.vscode-ruby-test-adapter-0.8.0/custom_formatter.rb --format CustomFormatter --order defined --dry-run

[2020-12-16 23:43:45.216] [ERROR] Error while finding RSpec test suite: Command failed: bundle exec rspec --pattern './spec/**/*_test.rb,./spec/**/test_*.rb,./spec/**/*_spec.rb' --require /Users/pbenjamin/.vscode/extensions/connorshea.vscode-ruby-test-adapter-0.8.0/custom_formatter.rb --format CustomFormatter --order defined --dry-run

env: ruby_executable_hooks: No such file or directory

When I run the command manually:

$ bundle exec rspec --pattern './spec/**/*_test.rb,./spec/**/test_*.rb,./spec/**/*_spec.rb' --require /Users/pbenjamin/.vscode/extensions/connorshea.vscode-ruby-test-adapter-0.8.0/custom_formatter.rb --format CustomFormatter --order defined --dry-run

Name                                                                                                           | Status | Time    |
-----------------------------------------------------------------------------------------------------------------------------------
START_OF_TEST_JSON{"version":"3.9.2","messages":[ <REDACTED> ],"examples":[],"summary":{"duration":5.7e-05,"example_count":0,"failure_count":0,"pending_count":0,"errors_outside_of_examples_count":2},"summary_line":"0 examples, 0 failures, 2 errors occurred outside of examples"}END_OF_TEST_JSON
Coverage report generated for RSpec to <REDACTED>/coverage/coverage.xml
Coverage report generated for RSpec to <REDACTED>/coverage. 686 / 3366 LOC (20.38%) covered.
SimpleCov failed with exit 1

[maxlorenz]

@pbnj I remember that I patched it by restricting the input to the first line of output of bundle. Was a one line change

[pbnj]

@maxlorenz - do you happen to have more information about this patch? Did you patch the vscode extension or did you patch some project configuration file?

[maxlorenz]

I patched the json reading function of the extension itself to only read the first line

[nicolaseckhart]

I have the same issue. When configuring SimpleCov to fail when below minimum coverage, the test detection fails for me due to low coverage.

# .simplecov

SimpleCov.start 'rails' do
  add_filter 'app/jobs/application_job.rb'
  add_filter 'app/models/application_record.rb'
  add_filter 'app/controllers/application_controller.rb'
end
SimpleCov.minimum_coverage 100

The above config produces the following error on launch:

[2021-02-08 13:32:47.387] [ERROR] Error while finding RSpec test suite: Command failed: bundle exec rspec --pattern './spec//**/*_test.rb,./spec//**/test_*.rb,./spec//**/*_spec.rb' --require /home/me/.vscode-server/extensions/connorshea.vscode-ruby-test-adapter-0.8.0/custom_formatter.rb --format CustomFormatter --order defined --dry-run
Line coverage (47.36%) is below the expected minimum coverage (100.00%).
SimpleCov failed with exit 2 due to a coverage related error

[simmerz]

We have a working solution until this gets resolved here.

First, we already don't auto-include simplecov, so gemfile contains gem "simplecov", require: false.

Then we add the following line to the top of our rails_helper.rb (could be spec_helper.rb if not using Rails):

require 'simplecov' unless ARGV.include?('--dry-run')

Which stops simplecov being included at all while the extension looks for the list of tests.