Python-Tls-Client icon indicating copy to clipboard operation
Python-Tls-Client copied to clipboard

memory leak

Open xiamoplus opened this issue 1 year ago • 6 comments

Hello, I saw that the memory leak was fixed by a recent pull request, but I still get an error from it. Any idea on how to avoid it? thx

xiamoplus avatar Apr 08 '23 13:04 xiamoplus

Update lib pip install --force-reinstall tls-client

OxynDev avatar Apr 09 '23 14:04 OxynDev

Also, having a memory leak issue. Can't seem to fix it :/

DaBossCoda avatar Apr 16 '23 07:04 DaBossCoda

The memory leak is still present. The issue can be reproduced with the following code. I have hosted a local httpbin server and sent some large random data as JSON to the library. The memory in the first iteration is around 36 MB which gradually increases to more than 250MB.

import tls_client

from memory_profiler import profile

@profile
def send_requests():
    session = tls_client.Session(
        client_identifier="chrome112",
        random_tls_extension_order=True
    )
    url = 'http://0.0.0.0:8080/anything'
    random_data = ["a"] * 10 ** 4
    res = session.post(url=url, json=random_data)
    json = res.json()
    print(len(json))

for i in range(100000):
    send_requests()

sumeshmurali avatar Oct 23 '23 06:10 sumeshmurali

@sumeshmurali Haven't checked how tls_client handles session closing, but locally I was able to reproduce the issue with your code. Memory for me increased from 55MB -> 283MB (10000) requests. However, after adding session close the memory seems to stabilize. Could you try it on your side and confirm or deny if this works?

Edit: After testing it on a real and relatively heavy page (15MB, used GET instead of POST) the leak seems to still be present even with session.close().

import tls_client

from memory_profiler import profile

@profile
def send_requests():
    session = tls_client.Session(
        client_identifier="chrome112",
        random_tls_extension_order=True
    )
    url = 'http://localhost:8000/anything'
    random_data = ["a"] * 10 ** 4
    try:
        res = session.post(url=url, json=random_data)
        print(res.status_code)
    finally:
        session.close()

for i in range(10000):
    send_requests()

kastger avatar Jul 05 '24 13:07 kastger

Try add force_http1 tls_client.Session(client_identifier="firefox119",force_http1=True) In golang version it help me solv this issue

OxynDev avatar Jul 16 '24 17:07 OxynDev

@OxynDev the issue being talked about is a memory leak. forcing the client to use http/1.1 would work. just not smart if you're trying to replicate a browser which this client is made for. depending on what backend you're sending these requests to. most will support this proto, but likely stuff wont work as intended. as servers will expect a browser to do http requests over newer protos

YABOIpy avatar Aug 11 '24 21:08 YABOIpy