pydis icon indicating copy to clipboard operation
pydis copied to clipboard

Benchmark Memory Usage Too

Open boramalper opened this issue 5 years ago • 2 comments

We should benchmark the memory usage as well, since it's an undoubtedly very important metric for an in-memory database.

boramalper avatar Mar 02 '19 18:03 boramalper

I love python, but this will most likely be the hardest part. Python bloats memory to much that in those cases Cython or C might be the answer even if it was slower than python. See this post for some basic python 2 memory usage. http://cerebralmanifest.com/python-memory-footprint/

pykler avatar Mar 03 '19 04:03 pykler

@pykler CPython's dictionary implementation is one of its most optimized parts, especially on Python 3+. Not just because dictionaries are very useful, but especially since it's used to back most of the objects in the system, too (__dict__). Python dictionaries are very well behaved, and will only be about half-to-two-thirds full once they're big enough to matter. Read more here: https://utcc.utoronto.ca/~cks/space/blog/python/DictionarySpaceUsage

Frankly it's probably way better in 2019 than it was 2011, tbh: https://www.youtube.com/watch?v=npw4s1QTmPg

mahmoud avatar Mar 06 '19 05:03 mahmoud