Coverage analysis under rails 8 beta seems to not be working

[gnarfle]

I recently updated an app to the rails 8 beta to play around with it and found the simplecov analysis reports a lot of missing code coverage. Attached is an example of a controller where the rails 7 version has 100% coverage and the rails 8 version says a lot of coverage is missing. These controllers are slightly reworked but the baffling part to me is that the rails 8 version declares I'm missing coverage for things like @user = User.new and params.permit not to mention saving the user, and I definitely have request specs and feature specs covering new and create here that get me 100% coverage in the rails version.

[Cosmo]

We have the same issue. As a quick workaround, we disabled parallelize(workers: :number_of_processors) in test/test_helper.rb.

[Dandush03]

here is my file, it's working fine with parallelize processes

#=> test/test_helper.rb
# frozen_string_literal: true

ENV['RAILS_ENV'] ||= 'test'

require_relative 'supports/simplecov'
require_relative '../config/environment'
require 'rails/test_help'

module ActiveSupport
  class TestCase
    I18n.locale = :en
    # Run tests in parallel with specified workers

    parallelize_setup do |_worker|
      SimpleCov.command_name "Job::#{Process.pid}" if const_defined?(:SimpleCov)
    end

    parallelize(workers: :number_of_processors)

    # Setup all fixtures in test/fixtures/*.yml for all tests in alphabetical order.
    fixtures :all

    # Add more helper methods to be used by all tests here...
  end
end

#=> test/supports/simplecov.rb
# frozen_string_literal: true

require 'simplecov'

Rake.respond_to?(:application) &&
  Rake.application.respond_to?(:top_level_tasks) &&
  Rake.application.top_level_tasks.any? { |e| e.eql?('test:prepare') }

FileUtils.rm_f('coverage/.resultset.json')

SimpleCov.minimum_coverage 100
SimpleCov.start(:rails) do
  add_filter do |source_file|
    removed_lines = 0
    relevant_lines = source_file.src.count do |line|
      line = line.strip # remove leading and trailing whitespaces
      next if line.empty? # skip empty lines
      next if line.start_with?('#') # skip comments
      next removed_lines += 1 if line.start_with?('class', 'module') # skip class, module declarations
      next if line == 'end' # skip ends

      true
    end
    relevant_lines -= removed_lines
    relevant_lines <= 1
  end

  add_filter 'vendor/gems'
  add_filter 'preview/'

  at_exit do
    SimpleCov.formatter = SimpleCov::Formatter::SimpleFormatter
    SimpleCov.result.format!
  end
end

[dpneumo]

Thanks Dandush03. Works well for me on Rails 8. I found the first filter awkward for my brain so I rewrote it to make it easier for me. Perhaps others will like this as well.

#=> test/supports/simplecov.rb
# frozen_string_literal: true

require 'simplecov'

Rake.respond_to?(:application) &&
  Rake.application.respond_to?(:top_level_tasks) &&
  Rake.application.top_level_tasks.any? { |e| e.eql?('test:prepare') }

FileUtils.rm_f('coverage/.resultset.json')

SimpleCov.minimum_coverage 80

SimpleCov.start(:rails) do
  # Ignore files without functional code 
  add_filter do |source_file|
    source = source_file.src
    ignored = source.reduce(0) do |ignored, line|
      case line.strip
      when /^$/, /^#/, /^class/, /^module/, /^end/
        ignored += 1
      end
      ignored
    end
    (source.count - ignored) <= 0 # When true, source_file is ignored
  end

  add_filter 'vendor'
  add_filter 'preview'
  add_filter 'test'
  add_filter 'spec'

  add_group 'Policies', 'app/policies'

  at_exit do
    SimpleCov.formatter = SimpleCov::Formatter::SimpleFormatter
    SimpleCov.result.format!
  end
end

Btw: I think the test should be (source.count - ignored) <= 0 rather than <= 1. That allows files such as app/helpers/application_helper.rb to be counted. That file looks like this for me:

module ApplicationHelper
  include Pagy::Frontend
end

Since I added the Pagy reference I think tests that touch that line should be counted. Am willing to be convinced otherwise.

[ccuilla]

This may be helpful for some.

I'm on Rails 8.1.0. New-ish app (started on 8.0.3). Just added SimpleCov. It was reporting 0% coverage over everything. That was definitely wrong. I commented out parallelize(workers: :number_of_processors) in test\test_helper.rb and now it measures coverage perfectly.

Hope this helps someone.