search icon indicating copy to clipboard operation
search copied to clipboard

UnicodeDecodeError when the raw query contains non ASCII characters

Open dannymilsom opened this issue 6 years ago • 0 comments

Using thor in conjunction with Django Rest Framework and with debug=True, the raw_query key/value pair is added to the response of any paginated queries. However we are not currently handling encoding/decoding correctly as the raw_query value raises a UnicodeDecodeError when the DRF JSONRenderer tries to serialize the JSON response (if the raw_query string contains non ASCII characters).

Probably this is because we are encoding the string at the wrong time - see the search.Query class

def __unicode__(self):
    """This is how we get to the actual underlying querystring"""
    return unicode(self.build_query()).encode('utf-8')

And then we try to encode AGAIN in the rendered - which raises the exception. We should instead explicitly only encode it when required - e.g. passing the query into the GAE search API (which is why I think this was added).

dannymilsom avatar Aug 22 '18 19:08 dannymilsom