rspec-core icon indicating copy to clipboard operation
rspec-core copied to clipboard

Interaction with focus and --only-failures

Open BestFriendChris opened this issue 8 years ago • 5 comments

Hi there. I just noticed something odd with the interaction between the focus tag and --only-failures.

My rspec config is set up like so:

RSpec.configure do |config|
  config.filter_run_when_matching :focus
  config.example_status_persistence_file_path = 'tmp/spec_results.txt'
end

In a few of my specs, I'm using fdescribe to only run a subset of my entire suite.

It appears that when using rspec --only-failures in this scenario, all of the focused specs are running, not just the failed ones.

$ rspec --only-failures
Run options:
  include {:focus=>true, :last_run_status=>"failed"}

Randomized with seed 5034
...F.

When I change all of the fdescribes to describe it works as expected.

$ rspec --only-failures
Run options:
  include {:last_run_status=>"failed"}

Randomized with seed 50880
F

BestFriendChris avatar Mar 31 '17 02:03 BestFriendChris

@myronmarston do we need to clear other meta data when using --only-failures?

JonRowe avatar Mar 31 '17 03:03 JonRowe

@myronmarston do we need to clear other meta data when using --only-failures?

Nope. It has nothing to do with --only-failures or focusing per-se; it's simply that we have been ORing multiple filters (e.g. running any examples that are focused OR failed on the last run). The fix is to AND multiple filters (e.g. run only examples that are focused AND failed on the last run).

See #2408 for my fix.

myronmarston avatar Mar 31 '17 04:03 myronmarston

Has there been any movement on this since @myronmarston made his PR? I think I'm running into this when I'm trying to combine --only-failures with targeting a specific context by line number. Untested example intended for demo purposes:

a file named "spec/array_spec.rb" with:

RSpec.describe 'Array' do
  context "all array tests" do
      it "checks for inclusion of 3" do
        expect([1, 2]).to include(3) # failure
      end

      it "checks for inclusion of 1" do
        expect([1, 2]).to include(1)
      end

      it "checks for inclusion of 2" do
        expect([1, 2]).to include(2)
      end
  end
end

With the above:

  • rspec spec/array_spec.rb gives "3 examples, 1 failure"
  • rspec spec/array_spec.rb --only-failures gives "1 example, 1 failure"
  • rspec spec/array_spec.rb:2 --only-failures gives "3 examples, 1 failure"

stevenaw avatar Sep 26 '19 15:09 stevenaw

You're encountering a slightly different "feature", in that specific line numbers take precedence over other filters, you are instructing RSpec to run that example group.

To answer your question though as you can see from the PR theres been no progress however as it requires a new feature, we can't change the existing behaviour as it would be a breaking change and in my opinion its arbitrary which is correct anyway.

JonRowe avatar Sep 26 '19 15:09 JonRowe

Ok thanks for taking the time to explain and summarize @JonRowe

stevenaw avatar Sep 26 '19 16:09 stevenaw

Closing because this is essentially a duplicate of rspec/rspec#10 the lack of ability to or metadata, if that every gets built this would be fixed by that.

JonRowe avatar Nov 27 '24 20:11 JonRowe