faiss icon indicating copy to clipboard operation
faiss copied to clipboard

fix swig ownership when assigning to struct pointer field

Open mdouze opened this issue 2 years ago • 5 comments

Summary: When faiss is called from SWIG, it is usually assumed that Python will do all the memory management, therefore assigning to a pointer in C++ should not transfer ownerhsip (default SWIG behavior). This diff fixed that.

Differential Revision: D51529085

mdouze avatar Nov 22 '23 16:11 mdouze

This pull request was exported from Phabricator. Differential Revision: D51529085

facebook-github-bot avatar Nov 22 '23 16:11 facebook-github-bot

hopefully this will finally fix https://github.com/facebookresearch/faiss/issues/2996

mdouze avatar Nov 22 '23 16:11 mdouze

@mdouze do we still need this one?

junjieqi avatar May 23 '24 17:05 junjieqi

Even though the final assert fails since the ownership disappears, it seems like the memory does not increase. This typemap looks right to me when I test it. We can discuss on next steps.

mnorris11 avatar Jul 31 '24 19:07 mnorris11

Current status after discussion:

ASAN does not seem to detect these Python memory leaks. There are no ASAN errors in a simple repro unit test like

def test_ownership_2(self):
        d = 32
        quantizer = faiss.IndexFlatL2(d)
        quantizer.this.own(False)

The destructor of IndexFlatL2 does not get called when quantizer.this.own(False). Removing this line will call the destructor.

We should introduce some Python unit test to catch these memory leaks.

mnorris11 avatar Aug 12 '24 17:08 mnorris11