remove password from log messages

[e-maud]

For security reasons, would it be possible to anonymized the URL in log messages of send_request, i.e. removing at least the password?

[acdha]

I think you could configure a logging filter for that to avoid the password being logged. Out of curiosity, have you looked at what requests/urllib3 is logging? I'm wondering whether the easiest fix would simply be to remove https://github.com/django-haystack/pysolr/blob/c24036fc686bd1911e412ac251e76ba8373f7a7a/pysolr.py#L383 and https://github.com/django-haystack/pysolr/blob/c24036fc686bd1911e412ac251e76ba8373f7a7a/pysolr.py#L427 but I guess we'd still need to redact usage in exceptions. This feels like something which requests/urllib3 should have something reusable since it's fairly common.

[e-maud]

Thanks for the answer.

I would not remove L383 and 427 as it is exactly those ones I need to monitor the indexing (!) More precisely, I am using the status return code.

I did log requests/urllib:

import http.client
http.client.HTTPConnection.debuglevel = 1
logging.getLogger("requests.packages.urllib3").setLevel(logging.INFO)

but it's impossible to read as it consists in the (big) list of docs to index. However I noticed that the password was somehow encrypted, and double-checked here: https://github.com/psf/requests/blob/a4c18cd733f97b5659a29589432d8a39e7a0de87/requests/auth.py#L66, where b64encode is indeed used.

One option would be use regex to clean up the pass (to be tested thoroughly, though):

import re
url = "https://myuser:[email protected]/solr/collection/update/"
new_url = re.sub('(https://[^:]+:)([^@]+)(@.+)', r'\1password\3', s)
=> 'https://myuser:[email protected]/solr/collection/update/'

[acdha]

I was thinking it might be safest to urlparse it when the Solr object is instantiated and use that to construct a sanitized URL which could be used in the logging / exception paths.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.