Better reporting for "dict subset" comparison

[nikitagashkov]

Preface

Python has a way of checking that one dict is a subset of another one, as highlighed in https://github.com/pytest-dev/pytest/issues/2376#issuecomment-338407594. This is quite a nifty feature that allows for partial checks when you don't want to check all the fields e.g., some of them are not stable, so you want to split the checks:

def test_unstable():
    dict = {"a": 42, "b": random.random()}

    assert dict.items() >= {"a": 42}.items()
    assert 0 < dict["b"] <= 1

or the dict has too many irrelevant items and you don't want to list them all just to check a couple of interesting ones:

def test_irrelevant():
    dict = requests.get("https://example.com/big-json-with-lots-of-fields")

    assert dict.items() >= {"a": 42, "b": "a113"}.items()

What's the problem this feature will solve?

Described approach works perfectly fine from the functional standpoint however since there is no dedicated handling of this case, reporting fallbacks to repr of the ItemsView:

    def test_big_left():
        left = {"a": 1, "b": 2, "c": 3, "d": 4}
        right = left.copy() | {"e": 5}

>       assert left.items() >= right.items()
E       AssertionError: assert dict_items([('a', 1), ('b', 2), ('c', 3), ('d', 4)]) >= dict_items([('a', 1), ('b', 2), ('c', 3), ('d', 4), ('e', 5)])
E        +  where dict_items([('a', 1), ('b', 2), ('c', 3), ('d', 4)]) = <built-in method items of dict object at 0x103a65bc0>()
E        +    where <built-in method items of dict object at 0x103a65bc0> = {'a': 1, 'b': 2, 'c': 3, 'd': 4}.items
E        +  and   dict_items([('a', 1), ('b', 2), ('c', 3), ('d', 4), ('e', 5)]) = <built-in method items of dict object at 0x103a65ec0>()
E        +    where <built-in method items of dict object at 0x103a65ec0> = {'a': 1, 'b': 2, 'c': 3, 'd': 4, ...}.items

It would be nice to have a dedicated comparison for this that shows only the difference, similar to dict comparison:

    def test_repr():
        left = {"a": 1, "b": 2, "c": 3, "d": 4}
        right = left.copy() | {"e": 5}

>       assert left == right
E       AssertionError: assert {'a': 1, 'b':...c': 3, 'd': 4} == {'a': 1, 'b':..., 'd': 4, ...}
E
E         Omitting 4 identical items, use -vv to show
E         Right contains 1 more item:
E         {'e': 5}
E         Use -v to get more diff

Describe the solution you'd like

Currently, I have a naive implementation for such case that utilizes pytest_assertrepr_compare hook:

def pytest_assertrepr_compare(
    config: Config,
    op: str,
    left: object,
    right: object,
) -> list[str] | None:
    if (
        isinstance(left, ItemsView)
        and isinstance(right, ItemsView)
        and
        # Naive implementation for superset comparison only. XXX: Support other
        # comparisons?
        op == ">="
    ):
        missing: list[str] = []
        differing: list[str] = []

        left_dict = dict(left)
        for k, v in right:
            # XXX: Shouldn't `k` and `v` go through `pytest_assertrepr_compare` as well?
            if k not in left_dict:
                missing.append(f"  {k!r}: {v!r}")
            elif left_dict[k] != v:
                differing.append(f"  {k!r}: {left_dict[k]!r} != {v!r}")
        assert missing or differing  # Otherwise, why are we even here?

        # XXX: Better header?
        output = ["left dict_items(...) is not a superset of right dict_items(...)"]
        if missing:
            output.append("Missing:")
            output.extend(missing)
        if differing:
            output.append("Differing:")
            output.extend(differing)

        return output

    return None

With this hook in place, the output looks like this:

    def test_repr():
        left = {"a": 1, "b": 2, "c": 3, "d": 4}
        right = left.copy() | {"e": 5}

>       assert left.items() >= right.items()
E       AssertionError: assert left dict_items(...) is not a superset of right dict_items(...)
E         Missing:
E           'e': 5

and for a mismatch:

    def test_repr():
        left = {"a": 1, "b": 2, "c": 3, "d": 4}
        right = left.copy() | {"a": 12}

>       assert left.items() >= right.items()
E       AssertionError: assert left dict_items(...) is not a superset of right dict_items(...)
E         Differing:
E           'a': 1 != 12

Alternative Solutions

I think it's possible to introduce rich comparison with third-party plugins that implement the hook similar to aforementioned or we can introduce some unittest-style assertDictIsSubset that handles reporting, but to be honest I'm leaning towards thinking that first-party support via assertrepr for this would be best.

Additional context

—

[jmehnle]

I like this, however relying on the got_dict.items() >= want_dict.items() idiom has the potentially substantial issue that it doesn't support nesting partial dicts in want_dict. So the utility of this idiom is somewhat limited, and the relevance of assertion introspection for specifically this idiom is therefore limited, too.

In my understanding a solution that deals well with nested partial dicts would need to involve attaching the assertion introspection behavior to the want_dict or any of its nested partial dicts, such as the anys package's AnyWithEntries dict wrapper. No pytest_assertrepr_compare implementation will ever be able to change actual comparison behavior, which however is needed to support nested partial dict comparisons.

E.g., this successfully does nested partial dict comparison, but the assertion error explanation is still impossible to read:

>       assert data == AnyWithEntries({
            'id': data_store_id,
            'project': AnyWithEntries({
                'id': project_id,
                'symbolic_name': 'my-project',
            }),
            'symbolic_name': 'data-store',
            'title': 'Data Store Title',
            'description': 'Mongo DB data store',
            'type': 'document',
        })
E       AssertionError: assert {'symbolic_name': 'data-store', 'title': 'Data Store Title', 'description': 'Data Store Description', 'type': 'sql', 'vendor': None, 'product': None, 'created_at': '2025-04-09T21:59:39.874208Z', 'updated_at': '2025-04-09T21:59:39.874208Z', 'id': 24, 'project': {'title': 'My Project', 'symbolic_name': 'my-project', 'description': None, 'created_at': '2025-04-09T21:59:39.846335Z', 'updated_at': '2025-04-09T21:59:39.846335Z', 'id': 157, 'owner_user': {'email': '[email protected]', 'name': 'Alice Awkward', 'verified': True, 'is_admin': False, 'organization_name': None, 'created_at': '2025-04-09T21:59:39.467295Z', 'updated_at': '2025-04-09T21:59:39.467300Z', 'id': 1}}} == AnyWithEntries({'id': 24, 'project': AnyWithEntries({'id': 157, 'symbolic_name': 'my-project'}), 'symbolic_name': 'data-store', 'title': 'Data Store Title', 'description': 'Mongo DB data store', 'type': 'document'})
E        +  where AnyWithEntries({'id': 24, 'project': AnyWithEntries({'id': 157, 'symbolic_name': 'my-project'}), 'symbolic_name': 'data-store', 'title': 'Data Store Title', 'description': 'Mongo DB data store', 'type': 'document'}) = AnyWithEntries({'id': 24, 'project': AnyWithEntries({'id': 157, 'symbolic_name': 'my-project'}), 'symbolic_name': 'data-store', 'title': 'Data Store Title', 'description': 'Mongo DB data store', 'type': 'document'})