badsecrets icon indicating copy to clipboard operation
badsecrets copied to clipboard

Replace requests with HTTPX

Open random-robbie opened this issue 1 year ago • 4 comments

Just as a heads up seen this while testing only errors during redirect.

badsecrets -u https://mail.pikselshop.com -r

 __ )              |                                |
 __ \    _` |   _` |   __|   _ \   __|   __|   _ \  __|   __|
 |   |  (   |  (   | \__ \   __/  (     |      __/  |   \__ \
____/  \__,_| \__,_| ____/ \___| \___| _|    \___| \__| ____/

v0.4.490

Traceback (most recent call last):
  File "/Users/rwiggins/miniconda3/bin/badsecrets", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/Users/rwiggins/miniconda3/lib/python3.11/site-packages/badsecrets/examples/cli.py", line 224, in main
    r_list = carve_all_modules(requests_response=res, custom_resource=custom_resource, url=args.url)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/rwiggins/miniconda3/lib/python3.11/site-packages/badsecrets/base.py", line 213, in carve_all_modules
    r_list = x.carve(**kwargs)
             ^^^^^^^^^^^^^^^^^
  File "/Users/rwiggins/miniconda3/lib/python3.11/site-packages/badsecrets/base.py", line 97, in carve
    cookies = dict(requests_response.cookies)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/rwiggins/miniconda3/lib/python3.11/site-packages/requests/cookies.py", line 334, in __getitem__
    return self._find_no_duplicates(name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/rwiggins/miniconda3/lib/python3.11/site-packages/requests/cookies.py", line 413, in _find_no_duplicates
    raise KeyError(f"name={name!r}, domain={domain!r}, path={path!r}")
KeyError: "name='skin', domain=None, path=None"

random-robbie avatar Dec 17 '24 17:12 random-robbie

it does seem to be very common on MailEnable Web Mail pages.

random-robbie avatar Dec 17 '24 17:12 random-robbie

This is definitely a problem with requests.

The move is probably to switch to httpx anyway, so we will see which happens first: i get around to converting to httpx, or requests fixes the issue.

liquidsec avatar Dec 18 '24 03:12 liquidsec

badsecrets.zip I did some changes... feel free to check it out and see if this is correct but it changes all requests to httpx and a docker container to test it.

random-robbie avatar Dec 18 '24 10:12 random-robbie

If you want to make a pull request I can take a look, however, this is going to be a very extensive operation as we have a lot of tests and they would all need to be converted as well

liquidsec avatar Dec 18 '24 12:12 liquidsec