assertEquals can't show a helpful diff and a provided message

[stuartlangridge]

Describe the bug

assertEquals (from std/testing) shows a nice diff when comparing two multiline strings. However, if an optional msg is passed to assertEquals then it shows the message but not the diff; there is no way to get both.

Steps to Reproduce

Obviously assertEquals could compare two arrays all in one go, but this is a simplified example. The key point is that a test may run more than one assert, but it's hard to know which one it failed on without outputting a message along with the assert, and then Deno.test suppresses the diff.

$ cat deno-assert-example.js
import { assertEquals } from "https://deno.land/[email protected]/testing/asserts.ts";

Deno.test("Nice diff, but I don't know which entry the compare failed on", () => {
    const list1 = ["abc\ndef", "ghi\jkl", "mno\npqr", "stu\nvwx"];
    const list2 = ["abc\ndef", "ghi\jkl", "mno\npqr", "AHA\nvwx"];
    for (let i=0; i<list1.length; i++) {
        assertEquals(list1[i], list2[i]);
    }
});

Deno.test("See the entry index number, but no nice diff", () => {
    const list1 = ["abc\ndef", "ghi\jkl", "mno\npqr", "stu\nvwx"];
    const list2 = ["abc\ndef", "ghi\jkl", "mno\npqr", "AHA\nvwx"];
    for (let i=0; i<list1.length; i++) {
        assertEquals(list1[i], list2[i], `failed on entry ${i}`);
    }
});

$ deno test deno-assert-example.js
running 2 tests from .../deno-assert-example.js
test Nice diff, but I don't know which entry the compare failed on ... FAILED (9ms)
test See the entry index number, but no nice diff ... FAILED (3ms)

failures:

Nice diff, but I don't know which entry the compare failed on
AssertionError: Values are not equal:

    [Diff] Actual / Expected

-   stu\n
+   AHA\n
    vwx

    at assertEquals (https://deno.land/[email protected]/testing/asserts.ts:177:9)

See the entry index number, but no nice diff
AssertionError: failed on entry 3
    at assertEquals (https://deno.land/[email protected]/testing/asserts.ts:177:9)

Expected behavior

It would be useful to be able to have assertEquals show a diff and the provided message, rather than only the provided message. (I personally would think that the diff could always be shown along with the message, but it's probably better to add an additional optional parameter includeDiffWithMessage=false or similar to assertEquals, in case people are passing huge strings which they are currently suppressing by providing a message.

Environment

• OS: Ubuntu 20.04
• deno version:
• deno 1.20.5 (release, x86_64-unknown-linux-gnu) v8 10.0.139.6 typescript 4.6.2
• std version: 0.149.0

[kt3k]

If we support the above, I'd suggest the overload like the below:

function assertEquals<T>(actual: T, expected: T, msg?: string);
function assertEquals<T>(actual: T, expected: T, msgFormatter: (msg: string) => string);

and the usage will be:

for (let i=0; i<list1.length; i++) {
  assertEquals(list1[i], list2[i], (msg) => `failed on entry ${i}: {msg}`);
}

[bajtos]

If we support the above, I'd suggest the overload like the below:

function assertEquals<T>(actual: T, expected: T, msg?: string);
function assertEquals<T>(actual: T, expected: T, msgFormatter: (msg: string) => string);

To make this API forward-compatible, I propose using an object instead of positional arguments in the formatter.

function assertEquals<T>(actual: T, expected: T, msgFormatter: (args: {diff: string}) => string);

Thoughts?

Also FWIW, Rust always prints both the message and the diff. Consider the following code:

assert_eq!(1, 2, "extra context");

When using assert_eq! from stdlib:

thread 'main' panicked at 'assertion failed: `(left == right)`
  left: `1`,
 right: `2`: extra context', src/main.rs:3:5

When using pretty_assertions crate used by Deno internally:

thread 'main' panicked at 'assertion failed: `(left == right)`: extra context

Diff < left / right > :
<1
>2

', src/main.rs:5:5

[bajtos]

Using https://pkg.go.dev/gotest.tools:

assert.Equal(t, val, 2, "extra context")

Produces error that includes both the diff and the custom message:

assertion failed: 1 (val int) != 2 (int): extra context

[kt3k]

Now I think showing both message and diff would be simpler and better than accepting custom formatter.

[bajtos]

I opened a PR implementing the changes discussed above: https://github.com/denoland/deno_std/pull/3253