yellowpages-scraper icon indicating copy to clipboard operation
yellowpages-scraper copied to clipboard

Was only able to get this to run once..

Open TechWithTy opened this issue 1 year ago • 1 comments

Was only able to get this to run once via google colab, the next day i tried im getting an error that the page failed to process.

!git clone https://github.com/scrapehero/yellowpages-scraper.git

!sudo apt-get install python3-lxml
!pip install requests lxml unicodecsv argparse

!ls

%cd yellowpages-scraper

!python3 yellow_pages.py restaurants Boston,MA

retrieving  https://www.yellowpages.com/search?search_terms=restaurants&geo_location_terms=Boston,MA
Failed to process page

TechWithTy avatar Jul 30 '23 13:07 TechWithTy

I think it may be an issue with bot detection and selenium web driver. will need to use undetectable driver and proxies to work.

TechWithTy avatar Jul 30 '23 14:07 TechWithTy