PyFunceble
PyFunceble copied to clipboard
FEATURE: Reputation filter using the proxy connection
Description
If you enables --reputation-lookup
&& --reputation
then they do not obey the proxy settings
File "/usr/lib/python3/dist-packages/requests/adapters.py", line 516, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='reputation.alienvault.com', port=443): Max retries exceeded with url: /reputation.data (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7fec793377d0>: Failed to establish a new connection: [Errno -2] Name or service not known'))
Log
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/home/joakim/.local/lib/python3.11/site-packages/PyFunceble/cli/processes/workers/base.py", line 395, in run
raise exception
File "/home/joakim/.local/lib/python3.11/site-packages/PyFunceble/cli/processes/workers/base.py", line 373, in run
result = self.target(consumed)
^^^^^^^^^^^^^^^^^^^^^
File "/home/joakim/.local/lib/python3.11/site-packages/PyFunceble/cli/processes/workers/tester.py", line 292, in target
.query_status()
^^^^^^^^^^^^^^
File "/home/joakim/.local/lib/python3.11/site-packages/PyFunceble/checker/base.py", line 161, in wrapper
return func(self, *args, **kwargs) # pylint: disable=not-callable
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/joakim/.local/lib/python3.11/site-packages/PyFunceble/checker/base.py", line 186, in wrapper
result = func(self, *args, **kwargs) # pylint: disable=not-callable
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/joakim/.local/lib/python3.11/site-packages/PyFunceble/checker/reputation/domain_and_ip.py", line 95, in query_status
result = query_object.query_status()
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/joakim/.local/lib/python3.11/site-packages/PyFunceble/checker/base.py", line 161, in wrapper
return func(self, *args, **kwargs) # pylint: disable=not-callable
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/joakim/.local/lib/python3.11/site-packages/PyFunceble/checker/base.py", line 186, in wrapper
result = func(self, *args, **kwargs) # pylint: disable=not-callable
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/joakim/.local/lib/python3.11/site-packages/PyFunceble/checker/reputation/base.py", line 309, in query_status
self.try_to_query_status_from_dns_lookup()
File "/home/joakim/.local/lib/python3.11/site-packages/PyFunceble/checker/reputation/base.py", line 198, in try_to_query_status_from_dns_lookup
if subject in self.ipv4_reputation_query_tool:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/joakim/.local/lib/python3.11/site-packages/PyFunceble/dataset/ipv4_reputation.py", line 77, in __contains__
with self.get_content() as file_stream:
^^^^^^^^^^^^^^^^^^
File "/home/joakim/.local/lib/python3.11/site-packages/PyFunceble/dataset/base.py", line 109, in wrapper
return func(self, *args, **kwargs) # pylint: disable=not-callable
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/joakim/.local/lib/python3.11/site-packages/PyFunceble/dataset/ipv4_reputation.py", line 100, in get_content
self.DOWNLOADER.start()
File "/home/joakim/.local/lib/python3.11/site-packages/PyFunceble/downloader/base.py", line 285, in start
if DownloadHelper(self.download_link).download_text(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/joakim/.local/lib/python3.11/site-packages/PyFunceble/helpers/download.py", line 235, in download_text
req = session.get(self.url, verify=self.certificate_validation)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 557, in get
return self.request('GET', url, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 544, in request
resp = self.send(prep, **send_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 657, in send
r = adapter.send(request, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/requests/adapters.py", line 516, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='reputation.alienvault.com', port=443): Max retries exceeded with url: /reputation.data (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7fbb2b7d3990>: Failed to establish a new connection: [Errno -2] Name or service not known'))
Possible Solution
make sure the global[rules] are used if set.
proxy:
global:
http: socks5h://127.0.0.1:9050
https: socks5h://127.0.0.1:9050
rules:
- http: socks5h://127.0.0.1:9050
https: socks5h://127.0.0.1:9050
tld:
- onion
Considered Alternative
rather boring as I would have to force the entire box to use proxy ala
alias proxyhttp="export http_proxy='http://$HOST:$PORT';export https_proxy='http://$HOST:$PORT'" # Squid
alias proxytor="export http_proxy='socks5h://localhost:9050';export https_proxy='socks5h://localhost:9050'" # Tor
alias proxyoff="unset http_proxy; unset https_proxy"
Additional context
Just stumbled on it
Thought:
In addition, optional use assigned --dns
, but should never be a default but a argument only
Just a status update on this: In the isuse-349
branch, a patch has been written.
Although it may work at runtime - at least on my machine, it doesn't pass the CI/CD tests because of circular import. So, I'll have to fix that first.
So, I'll have to fix that first.
... in rust :smile: As said, just stumbled on it as I was bored, not something I use. So guessing it isn't that important as nobody else has stumbled on it.