web_scraper icon indicating copy to clipboard operation
web_scraper copied to clipboard

Usage of ports in URL break validation in .loadWebPage()

Open Hecsall opened this issue 2 years ago • 0 comments

Doing some experiments I noticed that including a port (:XXXX) in the baseUrl inside webscraper.loadWebPage() method causes the validation to recognize it as a non-valid URL, and will throw a generic WebScraperException.

The following does not work

// These won't work
webscraper.loadWebPage('http://127.0.0.1:8080/')
webscraper.loadWebPage('https://somewebsite.com:9000/');

While those URLs don't work with webscraper.loadWebPage(), they still work with webscraper.loadFullUrl() due to missing validation:

// These will work
webscraper.loadFullUrl('http://127.0.0.1:8080/')
webscraper.loadFullUrl('https://somewebsite.com:9000/');

Hecsall avatar Aug 24 '21 19:08 Hecsall