Add proxy support for urllib3

[dliappis]

Currently it is possible to use elasticsearch-py behind a proxy by switching to the requests library, which honors proxy variables [1].

As discussed in https://github.com/elastic/rally/pull/584#pullrequestreview-167272980 it would be awesome to get proxy support for urllib3 as well.

[1] Related issues:

• https://github.com/elastic/elasticsearch-py/issues/357
• https://github.com/elastic/elasticsearch-py/issues/275#issuecomment-143682556

[dliappis]

FYI @fxdgear

[maharg101]

+1

[pquentin]

It's not really practical with urllib3 because in elastic_transport's Urllib3HttpNode uses urllib3's HTTPConnectionPool and HTTPSConnectionPool while proxies are handled at the PoolManager/ProxyManager level which is completely ignored here. However we can easily use requests. The difference with https://github.com/elastic/elasticsearch-py/issues/275#issuecomment-143682556 is that elasticsearch.Elasticsearch now raises a TypeError if we pass it an unknown kwarg, so we hardcode the proxies in CustomHttpNode:

from elastic_transport import RequestsHttpNode
from elasticsearch import Elasticsearch


class CustomHttpNode(RequestsHttpNode):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        # I launched an HTTP proxy using `python dummyserver/proxy.py` from the urllib3 code base
        self.session.proxies = {"https": "http://localhost:8888"}


es = Elasticsearch(
    "https://localhost:9200",
    # See https://www.elastic.co/guide/en/elasticsearch/client/python-api/current/connecting.html to get those values
    ssl_assert_fingerprint="725a0b3250ad15c01db1f31d81190490196038e2e51026bc9113ced2b36b0c7c",
    basic_auth=("elastic", "5w6VlIJxgygvweV66epH"),
    node_class=CustomHttpNode,
)
print(es.info().body, indent=2)

The downside of using requests is added complexity and lack of support for full HTTPS proxies (TLS-in-TLS): https://github.com/psf/requests/pull/5665.