memory leak in both fwd and inverse transforms

[kkothari93]

import numpy as np
import pyct as ct

img = np.random.rand(128,128)
A = ct.fdct2((128,128), 4, 16, False, norm=False, cpx=False)
c = np.random.rand(49577) #length of coeffs for 128x128 image

def check_leak_fwd(A, img):
  coeffs = np.zeros(49577)
  for i in range(5000):
    coeffs = A.fwd(img)

  return

def check_leak_inv(A, c):
  img = np.zeros((128, 128))
  for i in range(5000):
    img = A.inv(c)

  return 

if __name__=='__main__':
  check_leak_inv(A, c)

If you observe "free -h" on Ubuntu 18.04 with kernel 4.15.0-76-generic running python 3.6.8 while running the above file, you can clearly observe memory leak. If you have memory_profiler installed you can also run mprof run --include-children python <filename> and confirm the leak. I also see the leak in Python2.

[jtherrmann]

I'm having the same problem with Python 3.8.2 on Pop!_OS 20.04 LTS. I'll try to debug the problem further and see if I can learn anything useful.

[jtherrmann]

It looks like fdct2.fwd (defined in fdct2.py) calls fdct2_wrapper (defined in fdct2_wrapper.py) which calls _fdct2_wrapper.fdct2_wrapper, at which point the memory leak occurs. It looks like _fdct2_wrapper is a module imported from _fdct2_wrapper.cpython-38-x86_64-linux-gnu.so (a shared object file). I'm not familiar with the build steps for this project or how this file was generated, and I don't have experience debugging memory leaks in compiled code, so I'm stuck for now.

[jtherrmann]

Nevermind. It turns out my input array was just too large. My computer ran out of memory when I called fdct2.fwd on a 16384x16384 array but it worked fine when I called it on a 512x512 array. I should have checked the size of the image I was working with.