Unexpected result when querying MapField or DictField

[markf94]

When querying a MapField or DictField the order of the keys & values matter. After digging into the MongoDB documentation I understood that this has to do with the fact that MongoDB is using BSON to represent documents and BSON does care about the order of the keys & values (and generally even allows duplicate keys). However, mongoengine aims to be a Python Object-Document-Mapper and, hence, I think it should fulfill a Python user's expectation that is that the ordering of dict keys & values does not matter.

Here's a simple example implementation with Python 3.9 that demonstrates the problem:

from mongoengine import *


class OtherDocument(Document):
    name = StringField()


class MyMainDocument(Document):
    normal_dict_field = DictField()
    reference_dict_field = DictField(ReferenceField(OtherDocument))
    reference_map_field = MapField(ReferenceField(OtherDocument))


if __name__ == "__main__":

    connect()

    # define some "other" documents
    a = OtherDocument(name="a").save()
    b = OtherDocument(name="b").save()
    c = OtherDocument(name="c").save()
    document = MyMainDocument(
        normal_dict_field=dict(a="a", b="b", c="c"),
        reference_dict_field=dict(a=a, b=b, c=c),
        reference_map_field=dict(a=a, b=b, c=c),
    ).save()

    permutated_normal_dicts = [
        dict(a="a", b="b", c="c"),
        dict(b="b", c="c", a="a"),
        dict(c="c", a="a", b="b"),
    ]

    print("Querying normal_dict_field...")
    for dict_ in permutated_normal_dicts:
        if query_result := MyMainDocument.objects(normal_dict_field=dict_):
            print(f"Found {query_result} with {dict_}")
        else:
            print(f"Did not find any documents with {dict_}: {query_result}")

    permutated_reference_dicts = [
        {
            "a": OtherDocument.objects(name="a").first(),
            "b": OtherDocument.objects(name="b").first(),
            "c": OtherDocument.objects(name="c").first(),
        },
        {
            "b": OtherDocument.objects(name="b").first(),
            "a": OtherDocument.objects(name="a").first(),
            "c": OtherDocument.objects(name="c").first(),
        },
        {
            "a": OtherDocument.objects(name="a").first(),
            "c": OtherDocument.objects(name="c").first(),
            "b": OtherDocument.objects(name="b").first(),
        },
    ]

    print("Querying reference_dict_field...")
    for dict_ in permutated_reference_dicts:
        if query_result := MyMainDocument.objects(reference_dict_field=dict_):
            print(f"Found {query_result} with {dict_}")
        else:
            print(f"Did not find any documents with {dict_}: {query_result}")

    print("Querying reference_map_field...")
    for dict_ in permutated_reference_dicts:
        if query_result := MyMainDocument.objects(reference_map_field=dict_):
            print(f"Found {query_result} with {dict_}")
        else:
            print(f"Did not find any documents with {dict_}: {query_result}")

which prints:

Querying normal_dict_field...
Found [<MyMainDocument: MyMainDocument object>, <MyMainDocument: MyMainDocument object>, <MyMainDocument: MyMainDocument object>] with {'a': 'a', 'b': 'b', 'c': 'c'}
Did not find any documents with {'b': 'b', 'c': 'c', 'a': 'a'}: []
Did not find any documents with {'c': 'c', 'a': 'a', 'b': 'b'}: []
Querying reference_dict_field...
Found [<MyMainDocument: MyMainDocument object>] with {'a': <OtherDocument: OtherDocument object>, 'b': <OtherDocument: OtherDocument object>, 'c': <OtherDocument: OtherDocument object>}
Did not find any documents with {'b': <OtherDocument: OtherDocument object>, 'a': <OtherDocument: OtherDocument object>, 'c': <OtherDocument: OtherDocument object>}: []
Did not find any documents with {'a': <OtherDocument: OtherDocument object>, 'c': <OtherDocument: OtherDocument object>, 'b': <OtherDocument: OtherDocument object>}: []
Querying reference_map_field...
Found [<MyMainDocument: MyMainDocument object>] with {'a': <OtherDocument: OtherDocument object>, 'b': <OtherDocument: OtherDocument object>, 'c': <OtherDocument: OtherDocument object>}
Did not find any documents with {'b': <OtherDocument: OtherDocument object>, 'a': <OtherDocument: OtherDocument object>, 'c': <OtherDocument: OtherDocument object>}: []
Did not find any documents with {'a': <OtherDocument: OtherDocument object>, 'c': <OtherDocument: OtherDocument object>, 'b': <OtherDocument: OtherDocument object>}: []

[markf94]

For other people encountering this, the quick workaround is to query the individual fields of the MapField or DictField like this:

MyMainDocument.objects(normal_dict_field__a="a", normal_dict_field__b="b", normal_dict_field__c="c").first()