web_scraper
web_scraper copied to clipboard
Usage of ports in URL break validation in .loadWebPage()
Doing some experiments I noticed that including a port (:XXXX) in the baseUrl inside webscraper.loadWebPage()
method causes the validation to recognize it as a non-valid URL, and will throw a generic WebScraperException.
The following does not work
// These won't work
webscraper.loadWebPage('http://127.0.0.1:8080/')
webscraper.loadWebPage('https://somewebsite.com:9000/');
While those URLs don't work with webscraper.loadWebPage()
, they still work with webscraper.loadFullUrl()
due to missing validation:
// These will work
webscraper.loadFullUrl('http://127.0.0.1:8080/')
webscraper.loadFullUrl('https://somewebsite.com:9000/');