selenium-wire
selenium-wire copied to clipboard
wkeeling
memory does not last
Driving chrome with selenium-wire consumes incredible memory. constantly increasing memory consumption
Are you able to give some more details - the setup you're using plus the memory usage values you're seeing.
It's okay if I just detect driver.get BUT If I use it with an interceptor, the problem is big.
""def interceptor(request):
del request.headers['Referer']
request.headers['Referer'] = Refer ""
I change the global variable "Refer" with every "driver.get" command. yes the referer value changes but it uses incredible memory and finally "cant open new threat"
Are you using separate threads to run each driver.get? How long does the process run before the memory usage starts to get large, and how large does it get?
"""
from seleniumwire import webdriver
from selenium.webdriver.chrome.options import Options
import gc
i = 0
url = "https://www.google.com/"
chrome_options = Options()
chrome_options.add_argument("--headless")
chrome_options.add_argument("--no-sandbox")
chrome_options.add_argument("--disable-gpu")
chrome_options.add_argument("--disable-dev-shm-usage")
chrome_options.add_argument("--disable-browser-side-navigation")
chrome_options.add_argument("lang=zh_CN.UTF-8")
while True:
print("####### test driver %s #######" % i)
i += 1
driver = webdriver.Chrome(options=chrome_options, )
try:
driver.get(url)
source_body = driver.page_source
except Exception as e:
print(e)
finally:
driver.quit()
"""
When running this code on linux, you can observe that memory is constantly increasing.
But if i import package like this
""" from selenium import webdriver """, the problem is gone.
How I can fix this problem?
