botasaurus
botasaurus copied to clipboard
Can not download driver when running parallel
I run sample code and error has occurred:
selenium.common.exceptions.WebDriverException: Message: 'chromedriver-122' executable needs to be in PATH. Please see https://chromedriver.chromium.org/home
I think that system does not wait until downloading driver finish so it raise exception.
Here Code example:
from botasaurus import *
@browser(parallel=bt.calc_max_parallel_browsers, block_resources=True, block_images=True, data=["https://www.yahoo.com/", "https://www.google.com", "https://stackoverflow.com/"])
def scrape_heading_task(driver: AntiDetectDriver, data):
# print("metadata:", metadata)
print("data:", data)
# Navigate to the Omkar Cloud website
driver.get(data)
# Retrieve the heading element's text
heading = driver.text("h1")
title = driver.title
# Save the data as a JSON file in output/scrape_heading_task.json
return {
"heading": heading,
"title": title
}
if __name__ == '__main__':
scrape_heading_task()