aioresponses icon indicating copy to clipboard operation
aioresponses copied to clipboard

Wrongful (double) URL-encoding

Open mw-23 opened this issue 4 years ago • 1 comments

The function "aioresponses.core.normalize_url" appears to encode query arguments twice, such that e.g. @ is transform first to %40 and then wrongfully again to %2540 (since %25 is the encoding of a literal %).

The below example highlights this issue.

from urllib.parse import parse_qsl, urlencode, quote

from yarl import URL


def normalize_url(url: 'Union[URL, str]') -> 'URL':
    """Normalize url to make comparisons."""
    url = URL(url)
    return url.with_query(urlencode(sorted(parse_qsl(url.query_string))))


if __name__ == '__main__':
    base = 'http://test.com?abc='
    query_part = '@12'
    url1 = base + query_part
    url2 = URL(base + query_part)
    nurl1 = normalize_url(url1)
    nurl2 = normalize_url(url2)
    print(nurl1)  # incorrect encoding of @ == %2540
    print(nurl2)  # incorrect encoding of @ == %2540 
    print(base + quote(query_part))  # correct encoding of @ == %40

using python 3.8 and aioresponses version 0.7.1

mw-23 avatar Jan 02 '21 18:01 mw-23

Any comments on this, @pnuckowski ? Makes it a headache to get URLs to match sometimes.

JonasKs avatar Aug 11 '21 18:08 JonasKs