Diff reports confusing output when used with "fuzzy" matchers like `anything`
Subject of the issue
Using .to match with a complex expectation that mixes pure values with "fuzzy" matchers like anything produces false positives in Diff, obfuscating which key actually differs from the expectation.
I don't know whether this is a problem with match or anything/*_including.
Your environment
- Ruby version: 2.7.5 (also on 2.7.6)
- rspec-expectations version: 3.11.0
Steps to reproduce
# frozen_string_literal: true
begin
require "bundler/inline"
rescue LoadError => e
$stderr.puts "Bundler version 1.10 or later is required. Please update your Bundler"
raise e
end
gemfile(true) do
source "https://rubygems.org"
gem "rspec", "3.11.0"
gem "rspec-expectations", "3.11.0"
gem "rspec-mocks", "3.11.1"
end
puts "Ruby version is: #{RUBY_VERSION}"
require 'rspec/autorun'
RSpec.describe 'false positive in #match diff' do
it 'reports excess rows as mismatching' do
expect(
a: :a,
b: [:b, :bb],
c: :c,
d: :d,
).to match(
a: anything,
b: array_including(:b),
c: :c,
d: :actual_mismatch,
)
end
end
Output:
$ ruby rspec_demo.rb
Fetching gem metadata from https://rubygems.org/..
Resolving dependencies...
Using bundler 2.3.20
Using diff-lcs 1.5.0
Using rspec-support 3.11.1
Using rspec-core 3.11.0
Using rspec-expectations 3.11.0
Using rspec-mocks 3.11.1
Using rspec 3.11.0
Ruby version is: 2.7.5
F
Expected behavior
I'm expecting that Diff part will highlight only the key(s) actually failing their asserts, which is only :d.
Failures:
1) false positive in #match diff reports excess rows as mismatching
Failure/Error:
expect(
a: :a,
b: [:b, :bb],
c: :c,
d: :d,
).to match(
a: anything,
b: array_including(:b),
c: :c,
d: :actual_mismatch,
expected {:a=>:a, :b=>[:b, :bb], :c=>:c, :d=>:d} to match {:a=>#<RSpec::Mocks::ArgumentMatchers::AnyArgMatcher:0x00007ff34d667f18>, :b=>#<RSpec::Mocks::ArgumentMatchers::ArrayIncludingMatcher:0x00007ff34d58cc88 @expected=[:b]>, :c=>:c, :d=>:actual_mismatch}
Diff:
@@ -1,5 +1,5 @@
-:d => :actual_mismatch,
+:d => :d,
# rspec_demo.rb:23:in `block (2 levels) in <main>'
Actual behavior
Unfortunately, keys :a and :b are also highlighted, likely due to them using "fuzzy" matchers.
Failures:
1) false positive in #match diff reports excess rows as mismatching
Failure/Error:
expect(
a: :a,
b: [:b, :bb],
c: :c,
d: :d,
).to match(
a: anything,
b: array_including(:b),
c: :c,
d: :actual_mismatch,
expected {:a=>:a, :b=>[:b, :bb], :c=>:c, :d=>:d} to match {:a=>#<RSpec::Mocks::ArgumentMatchers::AnyArgMatcher:0x00007ff34d667f18>, :b=>#<RSpec::Mocks::ArgumentMatchers::ArrayIncludingMatcher:0x00007ff34d58cc88 @expected=[:b]>, :c=>:c, :d=>:actual_mismatch}
Diff:
@@ -1,5 +1,5 @@
-:a => anything,
-:b => array_including(b),
+:a => :a,
+:b => [:b, :bb],
:c => :c,
-:d => :actual_mismatch,
+:d => :d,
# rspec_demo.rb:23:in `block (2 levels) in <main>'
:wave: Hi, this is a known limitation of our differ which essentially only uses strings to produce diffs, so its unaware that the two sides did match it just sees the difference in the output used, theres not much we can do about it without a complete rewrite of the differ which is something we are interested in but has yet to find someone with enough time to finish it, I've transferred the issue to rspec-support which provides this functionality and renamed it slightly to better keep track of it, it related to rspec/rspec#94 and rspec/rspec#100.
Closing as part of the monorepo migration cleanup, there are several similar issues about improving the differ and we intend to make this pluggable.