frontera icon indicating copy to clipboard operation
frontera copied to clipboard

Handling encoding of URL's.

Open Preetwinder opened this issue 9 years ago • 1 comments

This problem arises when the URL is non-ascii. We can either store the URL's encoding but this introduces performance deterioration. Another option is to convert the URL's to an ascii representation such as - safe_url_string which is also used by scrapy, however this function also affects performance.

Preetwinder avatar Aug 03 '16 15:08 Preetwinder

It would be good to improve performance on that safe_url_string and use it on Request instantiation, probably.

sibiryakov avatar Aug 04 '16 09:08 sibiryakov