mongoengine icon indicating copy to clipboard operation
mongoengine copied to clipboard

`EmbeddedDocumentListField` causes additional memory usage.

Open neolooong opened this issue 1 year ago • 0 comments

This is the minimal example I found:

class Comment(EmbeddedDocument):
    content = StringField()

class Post(Document):
    title = StringField()
    comments = EmbeddedDocumentListField(Comment)

if Post.objects.count() == 0:
    for i in range(100):
        Post(title=f"{i}", comments=[Comment[f"{i}"]]).save()

def ok():
    qs = Post.objects
    for post in qs:
        post.title

def bug():
    qs = Post.objects
    for post in qs:
        post.comments

gc.collect()
ok()
print("collect after ok():", gc.collect())  # 0
bug()
print("collect after bug():", gc.collect())  # collect many objects

gc.collect() does not collect anything after exiting ok(). However, it can collect some objects after exiting bug(). The more times Post.comments accessed, the more memory used. Although this may not a big problem since it can eventually collect internally by python. In my case, this increase several gigabytes of RAM within a few minutes.

neolooong avatar Jun 20 '24 14:06 neolooong