Consider printing unified diff when it is small enough to highlight the diff between expected and actual results

[vlsi]

Feature summary

https://en.wikipedia.org/wiki/Diff#Unified_format

Well, my suggestion was behind the lines of https://en.wikipedia.org/wiki/Diff#Unified_format That is a very-well known format that is understandable by humans, allowing for highlighting the differences.

I think adding diff outputs would make certain failures easier to understand.

WDYT?

Example

Expecting actual:
  ["Yoda", "Han", "Luke", "Anakin"]
to contain exactly (and in same order):
  ["Yoda", "Luke", "Han", "Anakin"]
...
but there were differences at these indexes:
- index 1: actual element "Han" but expected was "Luke"
- index 2: actual element "Luke" but expected was "Han"
... OR
but there were differences at these indexes:
- element at index 1: expected "Han" but was "Luke"
- element at index 2: expected "Luke" but was "Han"

I would say "but there were differences at these indexes" is too wordy here, and it does take time to parse the output, understand what it wants to say, and so on.

In this case (comparing the iterables in order), a unified diff would work better, and it would produce the same style of messages no matter if the element is absent or extra.

Misplaced element
--- expected
+++ actual
@@ -1,4 +1,4 @@
 Yoda
-Han
 Luke
+Han
 Anakin

Unexpected element
--- expected
+++ actual
@@ -1,4 +1,4 @@
 Yoda
 Luke
+Han
 Anakin

Missing element
--- expected
+++ actual
@@ -1,4 +1,4 @@
 Yoda
-Han
 Luke
 Anakin

---

Example 2

See https://github.com/assertj/assertj-core/issues/2629#issue-1250479832

assertThat(Arrays.asList(1, 2, 3, 4)).containsExactly(1, 2, 4, 4);

Current AssertJ output:

java.lang.AssertionError: 
Expecting actual:
  [1, 2, 3, 4]
to contain exactly (and in same order):
  [1, 2, 4, 4]
but some elements were not found:
  [4]
and others were not expected:
  [3]

For instance, the output says "some elements were not found". Well, what does "4 were not found"? 4 is present in both actual and expected. Why does it tell me "4 were not found"? If I watch long enough, I could understand it wanted to say "you should have one more 4 in the output", however, I would say the current wording is misleading.

I believe, diff makes this case crystal clear: "you have 3 instead of 4 at position 3"

java.lang.AssertionError: 
Expecting actual:
  [1, 2, 3, 4]
to contain exactly (and in same order):
  [1, 2, 4, 4]
Here's the diff between expected and actual:
--- expected
+++ actual
@@ -1,4 +1,4 @@
 1
 2
-4
+3
 4

Example 3

assertThat(Arrays.asList(-1, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10)).containsExactly(0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 11);

Expecting actual:
  [-1, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
to contain exactly (and in same order):
  [0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 11]
but some elements were not found:
  [11]
and others were not expected:
  [-1, 9]

Here the diff shows the actual output has one extra -1 in the beginning, then it skips, and then it shows there are differences at the end (extra 9 and missing 11). It shows both "missing/extra" and the position of the deviation while the current "but some elements were not found" misses the context.

--- expected
+++ actual
@@ -1,3 +1,4 @@
+-1
 0
 1
 2
@@ -7,5 +8,5 @@
 6
 7
 8
+9
 10
-11

[joel-costigliola]

I'm not convinced this is super readable if you are not familiar with the diff format, we maybe could add a configuration property to replace the current error message with this format but I don't feel it should be the default.

--- expected
+++ actual
@@ -1,3 +1,4 @@
+-1
 0
 1
 2
@@ -7,5 +8,5 @@
 6
 7
 8
+9
 10
-11

[vlsi]

Well, I would say AssertJ users (developers?) should be pretty much familiar with diffs. Even GitHub UI shows that diff if you switch to "unified diff view"

[vlsi]

Here's another case for diff output: https://truth.dev/comparison#failure-messages. Note how diff output in XML makes it much easier to tell the difference.