Document instantiation performance

[kengruven]

This is probably a less common use case. I've got a mongoengine.Document that represents a common data structure in my application, which I store in my database, and it's great for that. But it's also a generally useful class, and in one place, we create it in an inner loop (to make use of its methods) without ever touching the database.

In this case, object instantiation can take up a non-trivial amount of time, especially compared to normal Python objects:

>>> timeit.timeit('Foo()', 'class Foo(object): pass', number=100000)
0.013084888458251953
>>> timeit.timeit('Bar()', 'import mongoengine\nclass Bar(mongoengine.Document): pass', number=100000)
2.403195858001709

Is there any way to make this more efficient? Or perhaps even to signal to mongoengine: "This is never going to be written to disk, so I don't really need all the metaclass stuff"?

[amcgregor]

I'd be curious what a comparison against marrow.schema:Container would be on the same hardware/setup as your first set of timeit tests. The overhead of a full ORM/ODM just to have a non-backed "data container" was the reason I wrote m.schema.

[kengruven]

I wrote this quick-n-dirty performance test to find out!

import sys
PY2 = (sys.version_info[0] == 2)

from timeit import timeit
N = 100000

tests = [{'module': 'builtins', 'class': 'object'},
         {'module': 'mongoengine', 'class': 'Document'},
         {'module': 'marrow.schema', 'class': 'Container'}]

if PY2:
    tests[0]['module'] = '__builtin__'

for test in tests:
    setup = 'import {module}\nclass Empty({module}.{class}): pass'.format(**test)
    test['time'] = timeit('Empty()', setup, number=N)
    print('{module}.{class}: {time}'.format(**test))

Results on my (slightly older) computer:

$ python2.7 perftest.py 
__builtin__.object: 0.013592004776
mongoengine.Document: 4.65804409981
marrow.schema.Container: 1.26947379112

$ python3.4 perftest.py 
builtins.object: 0.01261998899281025
mongoengine.Document: 4.775412261020392
marrow.schema.Container: 1.8198339309892617

So, 2.6-3.7x faster than MongoEngine, but still 100-150x slower than a plain object.