botasaurus icon indicating copy to clipboard operation
botasaurus copied to clipboard

KeyError: 'headers' at file reqs.py:323 [Possible solution suggested]

Open JimKarvo opened this issue 1 year ago • 0 comments

Hello,

the following code:

payload = {'var': 'true'}
response = request.post("https://mysite.com", data=payload, timeout=10)

produces the following error:

File "/usr/local/lib/python3.11/dist-packages/botasaurus_requests/reqs.py", line 365, in post
    fix_headers(kwargs)
    │           └ {'data': {'var': 'true'}, 'timeout': 10, 'allow_redirects': True, 'browser': 'firefox', 'proxy': 'http://x.x.x.x:xxxx'}
    └ <function fix_headers at 0x7f9c11eff600>
  File "/usr/local/lib/python3.11/dist-packages/botasaurus_requests/reqs.py", line 323, in fix_headers
    if isinstance(kwargs['headers'], dict):
                  └ {'data': {'var': 'true'}, 'timeout': 10, 'allow_redirects': True, 'browser': 'firefox', 'proxy': 'http://[72.10.160.172:16327'](x.x.x.x:xxxx')}

Seems like the code here, is not checks if the headers exists, look like it is mandantory to fil them. If i comment out the code, the botasaurus is working.

File: botasaurus_requests/reqs.py Line: 323

    if isinstance(kwargs['headers'], dict):
        # fix bug for ints not working
        for key, value in kwargs['headers'].items():
            # if isinstance(kwargs['headers'], int):
            
            kwargs['headers'][key] = str(value)

I think that the problem will be solved if we use the following:

headers = kwargs.get('headers')
if isinstance(headers, dict):

or:

if 'headers' in kwargs and isinstance(kwargs['headers'], dict):

JimKarvo avatar Aug 13 '24 10:08 JimKarvo