python-semrush icon indicating copy to clipboard operation
python-semrush copied to clipboard

display_filter is urlencoded by the Requests library

Open jroakes opened this issue 6 years ago • 0 comments

This repo, uses the requests library to hit the api URL and manage parameters. One issue with this is that when using display_filter, the api expects:

+|Ur|Co|electrical-supplies (include|URL|Containing|electrical-supplies)

to be:

%2B|Ur|Co|electrical-supplies

if you pass this string to the client.domain_organic method, it will be converted to:

%252B%7CUr%7CCo%7Celectrical-supplies

when Requests builds the URL request.

a workaround is to do the following:

url = "electrical-supplies"
display_filter = quote("+|Ur|Co|" + url, safe='|/', encoding=None, errors=None)    
client.api_url = 'http://api.semrush.com/?display_filter='+display_filter
data = client.domain_organic(domain="domain.com", database='us',display_limit=5,export_columns='Ur,Po,Ph,Nq')

The solution would be to simply have the retrieve method of the SemrushClient class handle a kwargs key of 'display_filter' directly on the api URL and let requests handle the remaining params.

There may be some other way to hook into urlencode, or the Requests library to skip encoding '|' characters, but I am not aware of any.

I realize this is a couple-year-old repo, so understood if not wanting to look at. I wanted to leave this here in case someone ran into the same issue.

jroakes avatar Feb 28 '18 22:02 jroakes