botasaurus
botasaurus copied to clipboard
websocket - ERROR - [Errno 61] Connection refused - goodbye with FastAPI
I'm getting this error when I try to run botasaurus along with FastAPI.
same issue here. any ideas on how to resolve?
I'm not using it with fastapi and getting this error still. My code:
@browser(block_images_and_css=True,
headless=True,
user_agent="Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36",
output=None,
reuse_driver=True,
close_on_crash=True,
parallel=2,
wait_for_complete_page_load=False)
def scrape(driver: Driver, data):
logging.info("Scraping " + data)
driver.google_get(data, bypass_cloudflare=True)
return driver.page_html
I would say this happens with a long list of links are told to be scraped. (Like 70-80 links)
I first use the browser decorator like the one below along with FastAPI and encounter the "Connetion refused" error
@browser(
│ output=None,
│ headless=True,
│ wait_for_complete_page_load=True,
│ create_error_logs=False,
│ add_arguments=[
│ │ "--disable-gpu",
│ │ "--no-sandbox",
│ │ "--disable-dev-shm-usage",
│ │ "--server",
│ │ "--disable-setuid-sandbox",
│ ],
)
But after I added these two lines, it was gone.
reuse_driver=True,
parallel=True,
I don't know what happened or how to fix but I hope this will help someone.