algolia-webcrawler
algolia-webcrawler copied to clipboard
Manage url that have non encoded characters
If the crawler tries to request a url that has non encoded characters, it will simply stop there and giv an error.
To fix this, url is first decoded to decoded what is already present in the url so that there wil not be a double encoding. It is then encoded.