BUG - python Uc object not freed even though no references are present

[paulkermann]

I have an object that wraps access to the Uc class. My emulator wrapper also registers hooks and such. To my surprise my python was getting the "Could not allocate dynamic translator buffer" which was weird because I was using the emulator only in a single function. By explicitly calling gc.collect the amount of emulator present were 2 but still, the object was out of scope and unused + calling the collect function was extremely slow.

I explicitly called delete on the Uc object but the memory allocated for it (the 1GB memory) was not freed and i was still getting that out of memory error. I tried searching who has references to the unicorn object so I found. I was able to get an instance of the Uc object from the _cleanup.refs. From there I called gc.get_referrers.

There I saw 3 references to the Uc object and those references were the hooks created for the emulator. Explicitly calling hook_del on my hooks did not have any effect on those references. It seems like the _catch_hook_exception was the only thing keeping a reference to the Uc Object because it wraps it with a self.

My current work around is a destroy_uc function which explicitly calls the release_handle function and zeros out the _uch of the Uc weakrefs. This way the memory emulator memory is freed "on demand" and when the reference count of those function wrappers is magically 0 then the finalizer callback does not do anything. However this issue should be fixed either way as the present only references are to himself. I am using the latest Unicorn version on a Windows 10 21H1 with python 3.10.0 64 bits

[github-actions[bot]]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 15 days.

[paulkermann]

hey, this issue was not fixed remove stale label or something

[wtdcode]

Hey, sorry I miss this issue. I remember this has been fixed somewhere? Could you give a small reproduction script?

[paulkermann]

sample_x86.py.txt I would assume that the reference count of the uc_wrapper class would be 0 after each test_x86_64 iteration. Also, manually running the garbage collection does clean those instances so their true reference count is 0. However with each iteration it seems like the class instances still remain.

[wtdcode]

If manually calling gc.collect works, I would assume it's a Python-related issue, not Unicorn's?

[paulkermann]

But unicorn does this whole weird UcCleanupManager thing while many other projects which also contain handles to c data do not do. Also, unicorn is the only one causing my python to crash...

[wtdcode]

But unicorn does this whole weird UcCleanupManager thing while many other projects which also contain handles to c data do not do. Also, unicorn is the only one causing my python to crash...

You are right. I definitely need to have a look at UcCleanupManager for 2.1.0 release as we plan to refactor the python bindings then.

Link to #1629

[elicn]

The new implementation handles resources more effeiciently. I used the following code to test this (this is a modified version of the code above):

# ... snip ...

import weakref

# weak refs to objects are collected here.
# those refs are removed automatically when the obj gets gc-ed
ws = weakref.WeakSet()

def test_x86_64():
    print("Emulate x86_64 code")

    my_wrapper = uc_wrapper()
    ws.add(my_wrapper.mu)
    my_wrapper.mu.emu_start(my_wrapper.address, my_wrapper.address + len(X86_CODE64))

    # now print out some registers
    # print(">>> Emulation done. Below is the CPU context")

    rax = my_wrapper.mu.reg_read(UC_X86_REG_RAX)
    # print(">>> RAX = 0x%x" %rax)


if __name__ == '__main__':
    while 1:
        test_x86_64()

        print(f'Ref count: {len(ws)}')
        time.sleep(1)

The current implementation keeps piling up expired Uc objects (note I removed the explicit call to gc), while the new implementation allow a fairly low number of such objects.

[github-actions[bot]]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 15 days.

[github-actions[bot]]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 15 days.

[github-actions[bot]]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 15 days.