pytest-icdiff icon indicating copy to clipboard operation
pytest-icdiff copied to clipboard

Locating a diff from context might be hard

Open Kaligule opened this issue 2 years ago • 1 comments

pytest-icdiff is awesome.

Problem

When testing small dicts it returns the full diff, but on big structures it only returns those parts that differ (see #25). Sometimes (in the case of big, nested dicts with similar parts in them) this makes it hard to locate where those diffs happen though. The context might not be enough to find where the difference is.

Example

The test


import copy

def big_structure(n):
    """Create a deeply nested tree of dicts"""
    if n == 0:
        return dict()
    return {
        "a": big_structure(n-1),
        "b": big_structure(n-1),
    }

def test_icdiffs():
    expected_structure = big_structure(6)

    real_structure = copy.deepcopy(expected_structure)
    real_structure["a"]["b"]["b"]["a"]["a"]["b"] = "BOOM!"  # create a difference

    assert expected_structure == real_structure

The result

Screenshot_2023-07-20_14:27:19

As you can see it is very hard (if even possible) to retrieve the path of the differing field (("a", "b", "b", "a", "a", "b")). This path is often is important to know to fix the error though

Suggested Solution

if pytest is called with -vvv (oder another appropriate level of verbosity) then the context could be disabled, showing the full diff.

Kaligule avatar Jul 20 '23 13:07 Kaligule

great idea! do you want to have a go at fixing it?

the place for the fix is near the end of the implementation:


    icdiff_lines = list(differ.make_table(pretty_left, pretty_right))
    if len(icdiff_lines) > 50:
        icdiff_lines = list(differ.make_table(pretty_left, pretty_right, context=True))

basically we'd want to add some sort of and not pytest.options.verbose=3 or whatever, to that if.

in terms of tests, you could start by copying test_really_long_diffs_use_context_mode and change the runpytest('-vv' bit to use -vvv...

hjwp avatar Sep 20 '23 22:09 hjwp