Trying to do a "join" on a ListField(ReferenceField(...)) doesn't raise an exception

[mlorant]

I have two models, let's say:

class Article(Document):
    title = StringField()
    tags = ListField(StringField())

class Blog(Document):
     articles = ListField(ReferenceField(Article))

I expected that Blog.objects.filter(articles__tags='lemon') would return the list of blogs that have at least one article using the tag lemon, but it just returns an empty set.

Is it the expected behaviour or a bug? Could it be supported?

Regards,

[lafrech]

I'm pretty sure this can't be done in ME.

This requires queries in two collections. Joins are MongoDB's shortcoming.

You could do this in two steps (do the join in application logic):

• get the list of all Articles with lemon tag,
• for each Article in the list, get the list of Blogs having Article in their articles list

Note that in this example, there is probably only one blog for each article, and you could add a blog field to Article to make this faster (one query). But I suppose your real use case is different, although a workaround could be to add a blogs Listfield to Article.

I'm no expert and you may want to ask on StackOverflow for confirmation if you don't get a definitive answer here.

If my understanding and what I say above is correct, then I agree this would look nice as a ME enhancement, but even if someone was willing to do it and if it was reasonably feasible, it could be argued that this would hide complexity too much and could let the user trigger highly consuming queries without realizing it. In other words, I don't expect this to happen any soon. However, assuming I'm right, MongoEngine should probably trigger a warning (an Exception) rather than failing silently.

[lafrech]

I would expect ME to trigger an exception

mongoengine.errors.InvalidQueryError: Cannot perform join in mongoDB

like in this example.

I'm marking this as a bug. I don't think this will be done in ME, but we should throw an exception rather than fail silently.

[amcgregor]

Actually, you can perform left outer joins in MongoDB, just using the aggregate pipeline $lookup stage.

[wojcikstefan]

Agreed Blog.objects.filter(articles__tags='lemon') should raise mongoengine.errors.InvalidQueryError instead of returning an empty list. This is most likely a bug in BaseDocument._lookup_field.

[icodeface]

I find the same question, I solve it like this:

stores = Store.objects(sId='9234894')
Comment.objects(store__in=stores)

[tinvaan]

Is the example provided in https://docs.mongoengine.org/guide/querying.html#filtering-queries for a different case?

# This will return a QuerySet that will only iterate over pages that have
# been written by a user whose 'country' field is set to 'uk'
uk_pages = Page.objects(author__country='uk')

This should/would throw an InvalidQuery right?