eland icon indicating copy to clipboard operation
eland copied to clipboard

eland_import_hub_model command failed to connect to Elastic cluster

Open FANGAreNotGnu opened this issue 2 years ago • 2 comments

I'm working on a AWS EC2 server. I followed the tutorial and started a multi-node cluster with docker-compose. Then followed this NER example trying to deploy a model to the the cluster.

But eland failed to connect to elastic.

If I use --url https://elastic:<myelasticcpassword>@localhost:9200, the error message is:

2022-07-12 19:26:44,527 INFO : Establishing connection to Elasticsearch
2022-07-12 19:26:44,530 WARNING : Node <Urllib3HttpNode(https://localhost:9200)> has failed for 1 times in a row, putting on 1 second timeout
2022-07-12 19:26:44,530 WARNING : Retrying request after failure (attempt 0 of 3)
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/elastic_transport/_transport.py", line 329, in perform_request
    meta, raw_data = node.perform_request(
  File "/usr/local/lib/python3.9/dist-packages/elastic_transport/_node/_http_urllib3.py", line 199, in perform_request
    raise err from None
elastic_transport.ConnectionError: Connection error caused by: NewConnectionError(<urllib3.connection.HTTPSConnection object at 0x7f5ce9683220>: Failed to establish a new connection: [Errno 111] Connection refused)
2022-07-12 19:26:44,531 WARNING : Node <Urllib3HttpNode(https://localhost:9200)> has failed for 2 times in a row, putting on 2 second timeout
2022-07-12 19:26:44,531 WARNING : Retrying request after failure (attempt 1 of 3)
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/elastic_transport/_transport.py", line 329, in perform_request
    meta, raw_data = node.perform_request(
  File "/usr/local/lib/python3.9/dist-packages/elastic_transport/_node/_http_urllib3.py", line 199, in perform_request
    raise err from None
elastic_transport.ConnectionError: Connection error caused by: NewConnectionError(<urllib3.connection.HTTPSConnection object at 0x7f5ce9683490>: Failed to establish a new connection: [Errno 111] Connection refused)
2022-07-12 19:26:44,532 WARNING : Node <Urllib3HttpNode(https://localhost:9200)> has failed for 3 times in a row, putting on 4 second timeout
2022-07-12 19:26:44,532 WARNING : Retrying request after failure (attempt 2 of 3)
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/elastic_transport/_transport.py", line 329, in perform_request
    meta, raw_data = node.perform_request(
  File "/usr/local/lib/python3.9/dist-packages/elastic_transport/_node/_http_urllib3.py", line 199, in perform_request
    raise err from None
elastic_transport.ConnectionError: Connection error caused by: NewConnectionError(<urllib3.connection.HTTPSConnection object at 0x7f5ce9683940>: Failed to establish a new connection: [Errno 111] Connection refused)
2022-07-12 19:26:44,532 WARNING : Node <Urllib3HttpNode(https://localhost:9200)> has failed for 4 times in a row, putting on 8 second timeout
Traceback (most recent call last):
  File "/usr/local/bin/eland_import_hub_model", line 190, in <module>
    es = get_es_client(args)
  File "/usr/local/bin/eland_import_hub_model", line 153, in get_es_client
    es_info = es_client.info()
  File "/usr/local/lib/python3.9/dist-packages/elasticsearch/_sync/client/utils.py", line 414, in wrapped
    return api(*args, **kwargs)
  File "/usr/local/lib/python3.9/dist-packages/elasticsearch/_sync/client/__init__.py", line 2277, in info
    return self.perform_request(  # type: ignore[return-value]
  File "/usr/local/lib/python3.9/dist-packages/elasticsearch/_sync/client/_base.py", line 286, in perform_request
    meta, resp_body = self.transport.perform_request(
  File "/usr/local/lib/python3.9/dist-packages/elastic_transport/_transport.py", line 329, in perform_request
    meta, raw_data = node.perform_request(
  File "/usr/local/lib/python3.9/dist-packages/elastic_transport/_node/_http_urllib3.py", line 199, in perform_request
    raise err from None
elastic_transport.ConnectionError: Connection error caused by: ConnectionError(Connection error caused by: NewConnectionError(<urllib3.connection.HTTPSConnection object at 0x7f5ce9683220>: Failed to establish a new connection: [Errno 111] Connection refused))

If I use --url https://elastic:<myelasticcpassword>@<ip_address_of_es01>:9200, the error message is:

2022-07-12 14:28:26,740 INFO : Establishing connection to Elasticsearch                                                                                                                                            
/usr/local/lib/python3.9/dist-packages/elasticsearch/_sync/client/__init__.py:395: SecurityWarning: Connecting to 'https://172.28.0.3:9200' using TLS with verify_certs=False is insecure                          
  _transport = transport_class(                                                                                                                                                                                    
2022-07-12 14:30:37,752 WARNING : Node <Urllib3HttpNode(https://172.28.0.3:9200)> has failed for 1 times in a row, putting on 1 second timeout                                                                     
2022-07-12 14:30:37,752 WARNING : Retrying request after failure (attempt 0 of 3)                                                                                                                                  
Traceback (most recent call last):                                                                                                                                                                                 
  File "/usr/local/lib/python3.9/dist-packages/elastic_transport/_transport.py", line 329, in perform_request                                                                                                      
    meta, raw_data = node.perform_request(                                                                                                                                                                         
  File "/usr/local/lib/python3.9/dist-packages/elastic_transport/_node/_http_urllib3.py", line 199, in perform_request                                                                                             
    raise err from None                                                                                                                                                                                            
elastic_transport.ConnectionError: Connection error caused by: NewConnectionError(<urllib3.connection.HTTPSConnection object at 0x7f9ab9082220>: Failed to establish a new connection: [Errno 110] Connection time$
 out)                                                                                                                                                                                                              
2022-07-12 14:32:48,824 WARNING : Node <Urllib3HttpNode(https://172.28.0.3:9200)> has failed for 2 times in a row, putting on 2 second timeout                                                                     
2022-07-12 14:32:48,824 WARNING : Retrying request after failure (attempt 1 of 3)                                                                                                                                  
Traceback (most recent call last):                                                                                                                                                                                 
  File "/usr/local/lib/python3.9/dist-packages/elastic_transport/_transport.py", line 329, in perform_request                                                                                                      
    meta, raw_data = node.perform_request(                                                                                                                                                                         
  File "/usr/local/lib/python3.9/dist-packages/elastic_transport/_node/_http_urllib3.py", line 199, in perform_request                                                                                             
    raise err from None                                                                                                                                                                                            
elastic_transport.ConnectionError: Connection error caused by: NewConnectionError(<urllib3.connection.HTTPSConnection object at 0x7f9ab9082490>: Failed to establish a new connection: [Errno 110] Connection time$
 out)                                                                                                                                                                                                              
2022-07-12 14:34:59,896 WARNING : Node <Urllib3HttpNode(https://172.28.0.3:9200)> has failed for 3 times in a row, putting on 4 second timeout                                                                     
2022-07-12 14:34:59,896 WARNING : Retrying request after failure (attempt 2 of 3)                                                                                                                                  
Traceback (most recent call last):                                                                                                                                                                                 
  File "/usr/local/lib/python3.9/dist-packages/elastic_transport/_transport.py", line 329, in perform_request                                                                                                      
    meta, raw_data = node.perform_request(                                                                                                                                                                         
  File "/usr/local/lib/python3.9/dist-packages/elastic_transport/_node/_http_urllib3.py", line 199, in perform_request                                                                                             
    raise err from None                                                                                                                                                                                            
elastic_transport.ConnectionError: Connection error caused by: NewConnectionError(<urllib3.connection.HTTPSConnection object at 0x7f9ab9082880>: Failed to establish a new connection: [Errno 110] Connection time$
 out)                                                                                                                                                                                                              
2022-07-12 14:37:10,968 WARNING : Node <Urllib3HttpNode(https://172.28.0.3:9200)> has failed for 4 times in a row, putting on 8 second timeout                                                                     
Traceback (most recent call last):                                                                                                                                                                                 
  File "/usr/local/bin/eland_import_hub_model", line 190, in <module>                                                                                                                                              
    es = get_es_client(args)                                                                                                                                                                                       
  File "/usr/local/bin/eland_import_hub_model", line 153, in get_es_client                                                                                                                                         
    es_info = es_client.info()                                                                                                                                                                                     
  File "/usr/local/lib/python3.9/dist-packages/elasticsearch/_sync/client/utils.py", line 414, in wrapped                                                                                                          
    return api(*args, **kwargs)                                                                                                                                                                                    
  File "/usr/local/lib/python3.9/dist-packages/elasticsearch/_sync/client/__init__.py", line 2277, in info
    return self.perform_request(  # type: ignore[return-value]
  File "/usr/local/lib/python3.9/dist-packages/elasticsearch/_sync/client/_base.py", line 286, in perform_request
    meta, resp_body = self.transport.perform_request(
  File "/usr/local/lib/python3.9/dist-packages/elastic_transport/_transport.py", line 329, in perform_request
    meta, raw_data = node.perform_request(
  File "/usr/local/lib/python3.9/dist-packages/elastic_transport/_node/_http_urllib3.py", line 199, in perform_request
    raise err from None
elastic_transport.ConnectionError: Connection error caused by: ConnectionError(Connection error caused by: NewConnectionError(<urllib3.connection.HTTPSConnection object at 0x7f9ab9082220>: Failed to establish a
new connection: [Errno 110] Connection timed out))

Adding/Removing the --insecure flag does not help. Do I need to run this NER example in cloud or is there something need to be changed in the default configuration file?

FANGAreNotGnu avatar Jul 12 '22 19:07 FANGAreNotGnu

+1

rschwabco avatar Nov 29 '22 22:11 rschwabco

+1

SurajViitk avatar Apr 13 '23 09:04 SurajViitk