memory leak after executing request

[averysadonion]

My program runs multiple threads of a function performing a get request, and although the number of threads remains constant throughout the program, the memory usage and object count gradually increases over time until no free memory is left on the system. After doing some digging I found that base requests library has some memory leaking issues with the response object that can be fixed by using .close() method on the session and response like so:

   r = session.get(url=url, headers=headers)
   r.close()
   session.close()

I can't seem to find a solution to solving this issue within this library.

Sample code:

def func1():
#populate list


while True:
        threads = []
        for item in list:
            t = threading.Thread(target=func2, args=(item, v, q))
            threads.append(t)
            t.start()
        for thread in threads:
            thread.join()
            print(f"Retrying... {retry}")
 
        retry += 1


def func2(i, v, q):
  headers = {'':''}
  
  session = Session(client_identifier='chrome_105')
  r = session.get('url', headers=headers)
  response = json.loads(r.text)
 #do stuff with response
 return

[FlorianREGAZ]

Hey, thats a problem in the golang library. We'll have to wait for bogdanfinn to fix it.

[fnk93]

@FlorianREGAZ the golang library already exposes the functions freeMemory, destroySession and destroyAll. The readme also tackles the issue of not freeing the responses here: (https://github.com/bogdanfinn/tls-client#compile-this-client-as-a-shared-library-for-use-in-other-languages-like-python-or-nodejs) With this, closing the response after retrieving all necessary values is already possible.

[pengyanbing84]

I have this problem. Is there a solution?

internal buffer error : Memory allocation failed : growing buffer fatal error: runtime: C malloc failed

[biggie0344]

#55 created a PR

[zyx898]

Is there a solution yet? Still has memory leaks