fluentassertions
collection Should().BeEquivalentTo() vs position of first difference
Description
I use the collection .Should().BeEquivalent() to ensure that two collections are equivalent. When my two collections do not have the same size, it is very hard to know at which index the first difference was found.
Complete minimal example reproducing the issue
Here's my test code:
class MyPair
{
public int x { get; set; }
public int y { get; set; }
public override string ToString() => $"{x}-{y}";
}
[Fact]
public void TestBeEquivalent()
{
var expected = Enumerable.Range(1, 100).Select(i => new MyPair { x = i, y = i + 20 }).ToList();
// Make 'actual' almost identical to 'expected'
var actual = new List<MyPair>(expected);
actual.Insert(95, new MyPair { x = 0, y = 0 });
actual.Should().BeEquivalentTo(expected);
}
Expected behavior:
I would expect the test to fail with a message similar to this that pin points where the two collections start to be non-equivalent:
The collections are identical until position 95 (0 based) where "0-0" is not equivalent to "96-116".
Actual behavior:
The test fails with this message:
Expected actual to be a collection with 100 item(s), but {1-21, 2-22, 3-23, 4-24, 5-25, 6-26, 7-27, 8-28, 9-29, 10-30, 11-31, 12-32, 13-33, 14-34, 15-35, 16-36, 17-37, 18-38, 19-39, 20-40, 21-41, 22-42, 23-43, 24-44, 25-45, 26-46, 27-47, 28-48, 29-49, 30-50, 31-51, 32-52, …69 more…}"
"contains 1 item(s) more than"
"{1-21, 2-22, 3-23, 4-24, 5-25, 6-26, 7-27, 8-28, 9-29, 10-30, 11-31, 12-32, 13-33, 14-34, 15-35, 16-36, 17-37, 18-38, 19-39, 20-40, 21-41, 22-42, 23-43, 24-44, 25-45, 26-46, 27-47, 28-48, 29-49, 30-50, 31-51, 32-52, …68 more…}.
As you can see, it is very hard to know "why" the collections are not equivalent. Just looking at the output, it is impossible to know the error. I have to run the debugger, watch variables, copy paste the two collections in a diff tool and find the differences from there. It would be very hepfull if FluentAssertions could help identify where the collections start to diverge.
Versions
- FluentAssertions: 5.5.3
- .NET Core 2.1
For anyone willing to pick up this task, be aware that comparing sequences with BeEquivalentTo can compare collections in either WithStrictOrdering() or WithoutStrictOrdering(), where the latter is the default.
The difference in strictness should be reflected in the failure message, as only strict ordering requires the sequence elements to match index by index.
Successful assertions should execute fast and failing assertions should provide good failure messages.
As pointed out, including everything in the failure message doesn't necessarily make a great failure message as it may hide the valuable information in the data. So the striking question for me here is, what information makes value to include in the failure message. Is it valuable to include all the matching elements or even any of the matching elements? If it does make sense to include some matching elements, could we at least do a more intelligent pruning to highlight the mismatch(es). E.g. only include one or more preceding and succeeding elements of the mismatch in the failure message.
Here are some sketches on how I think pruned failure message could look like.
WithoutStrictOrdering
var subject = new [] { 1, 2, 3, 4, 7, 8, 9, 42 };
var expected = new [] { 1, 2, 4, 5, 7, 8, 9, 43 };
subject.Should().BeEquivalentTo(expected, o => o.WithoutStrictOrdering());
Expected
{ ..., 2, 3, 4, ..., 9, 42}
to be equivalent to
{ ..., 2, 4, 5, ..., 9, 43},
but found no equivalents found for "3" (2) and "42" (7) in subject and "5" (3) and "43" (7) in expected.
WithStrictOrdering
var subject = new [] { 1, 2, 3, 4, 5, 6, 7 };
var expected = new [] { 1, 2, 42, 4, 5, 6, 43 };
subject.Should().BeEquivalentTo(expected, o => o.WithStrictOrdering());
Expected
{ ..., 2, 3, 4, ..., 6, 7}
to be equivalent to
{ ..., 2, 42, 4, ..., 6, 43},
but found mismatches between "3" (2) and "42" and "7" (6) and "43".
