python-idb icon indicating copy to clipboard operation
python-idb copied to clipboard

reduce cursor memory usage

Open williballenthin opened this issue 4 years ago • 1 comments

I also detected the single test method consumes 11GB memory, when running the tests with python2.7. Amazing!

@slow
@kern32_test()
def test_cursor_enum_all_asc(kernel32_idb, version, bitness, expected):
    minkey = kernel32_idb.id0.get_min().key
    cursor = kernel32_idb.id0.find(minkey)
    count = 1
    while True:
        try:
            cursor.next()
        except IndexError:
            break
        count += 1

    assert kernel32_idb.id0.record_count == count

Figure_test_idb_test_cursor_enum_all_asc_py27

Originally posted by @imbillow in https://github.com/williballenthin/python-idb/pull/81#issuecomment-659604158

williballenthin avatar Jul 21 '20 21:07 williballenthin

i suspect something like the page cache is not being used correctly and garbage is not getting free'd. need to go through the traversal code with a fine-toothed comb.

williballenthin avatar Jul 21 '20 21:07 williballenthin