elasticmock icon indicating copy to clipboard operation
elasticmock copied to clipboard

If change import style elasticsearch mock fuctionality not working properly

Open NagRaj48 opened this issue 3 years ago • 5 comments

import elasticsearch

class FooService:

def __init__(self):
    self.es = elasticsearch.Elasticsearch(hosts=[{'host': 'localhost', 'port': 9200}])

def create(self, index, body):
    es_object = self.es.index(index, body)
    return es_object.get('_id')

def read(self, index, id):
    es_object = self.es.get(index, id)
    return es_object.get('_source')

This works fine. But below code not working

from elasticsearch import Elasticsearch

class FooService:

def __init__(self):
    self.es = Elasticsearch(hosts=[{'host': 'localhost', 'port': 9200}])

def create(self, index, body):
    es_object = self.es.index(index, body)
    return es_object.get('_id')

def read(self, index, id):
    es_object = self.es.get(index, id)
    return es_object.get('_source')

================================================ getting errors like this

es_object = es.index(index, settings.config["es_index_jobs_mappings"])

python/tests/elastic_search/test_es_module.py:221:


../../../../my_env/local/lib/python3.8/site-packages/elasticsearch/client/utils.py:152: in _wrapped return func(*args, params=params, headers=headers, **kwargs) ../../../../my_env/local/lib/python3.8/site-packages/elasticsearch/client/init.py:398: in index return self.transport.perform_request( ../../../../my_env/local/lib/python3.8/site-packages/elasticsearch/transport.py:390: in perform_request raise e ../../../../my_env/local/lib/python3.8/site-packages/elasticsearch/transport.py:358: in perform_request status, headers_response, data = connection.perform_request(


self = <Urllib3HttpConnection: http://localhost:9200>, method = 'POST', url = '/xxxxxxxxxxxx/_doc', params = {} body = b'{}' timeout = None, ignore = (), headers = {}

def perform_request(
    self, method, url, params=None, body=None, timeout=None, ignore=(), headers=None
):
    url = self.url_prefix + url
    if params:
        url = "%s?%s" % (url, urlencode(params))
    full_url = self.host + url

    start = time.time()
    orig_body = body
    try:
        kw = {}
        if timeout:
            kw["timeout"] = timeout

        # in python2 we need to make sure the url and method are not
        # unicode. Otherwise the body will be decoded into unicode too and
        # that will fail (#133, #201).
        if not isinstance(url, str):
            url = url.encode("utf-8")
        if not isinstance(method, str):
            method = method.encode("utf-8")

        request_headers = self.headers.copy()
        request_headers.update(headers or ())

        if self.http_compress and body:
            body = self._gzip_compress(body)
            request_headers["content-encoding"] = "gzip"

        response = self.pool.urlopen(
            method, url, body, retries=Retry(False), headers=request_headers, **kw
        )
        duration = time.time() - start
        raw_data = response.data.decode("utf-8", "surrogatepass")
    except Exception as e:
        self.log_request_fail(
            method, full_url, url, orig_body, time.time() - start, exception=e
        )
        if isinstance(e, UrllibSSLError):
            raise SSLError("N/A", str(e), e)
        if isinstance(e, ReadTimeoutError):
            raise ConnectionTimeout("TIMEOUT", str(e), e)
      raise ConnectionError("N/A", str(e), e)

E elasticsearch.exceptions.ConnectionError: ConnectionError(<urllib3.connection.HTTPConnection object at 0x7f706a2b22b0>: Failed to establish a new connection: [Errno 111] Connection refused) caused by: NewConnectionError(<urllib3.connection.HTTPConnection object at 0x7f706a2b22b0>: Failed to establish a new connection: [Errno 111] Connection refused)

../../../../my_env/local/lib/python3.8/site-packages/elasticsearch/connection/http_urllib3.py:258: ConnectionError ----------------------------------------------------------- Captured stdout call ----------------------------------------------------------- <Elasticsearch([{'host': 'localhost', 'port': 9200}])>

NagRaj48 avatar Feb 26 '21 09:02 NagRaj48

Hi! Ty for thus comment!

Can't you change the way you import this? If not, do you have any change proposal to support both import styles?

I'm not actively maintaining this library, so it would be awesome if you send a PR with the changes to support that :)

vrcmarcos avatar Mar 02 '21 02:03 vrcmarcos

It wouldn't be possible to change production code as of now.

It would be better, the module must support below two import styles. ===> import elasticsearch

Then this would possible (elasticsearch.Elasticsearch(hosts=[{'host': 'localhost', 'port': 9200}]))

=========================== ====> from elasticsearch import Elasticsearch

then this would also possible(Elasticsearch(hosts=[{'host': 'localhost', 'port': 9200}]))

Kindly please add this feature in future additions. Thank you...:)

NagRaj48 avatar Mar 02 '21 07:03 NagRaj48

I think if you used patch.object(elasticsearch, "Elasticsearch", _get_elasticmock) instead of patch('elasticsearch.Elasticsearch', _get_elasticmock) in the elasticmock decorator, that would fix this. This is probably the cause of #82 as well.

RJPercival avatar Feb 06 '23 11:02 RJPercival

We cannot just change the way this is imported, because elasticsearch_dsl is importing it this way. I modified elasticsearch_dsl to change how it imports and this mocking library worked, but unfortunately that is not a viable solution for us to modify elasticsearch_dsl

matteius avatar Mar 09 '23 20:03 matteius

@RJPercival Unfortunately I just tried that change and was not able to get the mock to behave differently that way.

matteius avatar Mar 09 '23 20:03 matteius