BertClient(ip='localhost') hangs forever and never ends executing

[AndreaSottana]

Prerequisites

Please fill in by replacing [ ] with [x].

• [x] Are you running the latest bert-as-service?
• [x] Did you follow the installation and the usage instructions in README.md?
• [x] Did you check the FAQ list in README.md?
• [x] Did you perform a cursory search on existing issues?

System information

Some of this information can be collected via this script.

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Windows 10 64-bit
• TensorFlow installed from (source or binary): binary
• TensorFlow version: 1.15.0
• Python version: 3.7.9 ('tags/v3.7.9:13c94747c7', 'Aug 17 2020 18:58:18')
• bert-as-service version: 1.10.0
• GPU model and memory: No GPU available
• CPU model and memory: Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 15.8GB

---

Description

I'm using this command to start the server:

bert-serving-start -model_dir=/d/Users/andrea.sottana/Desktop/notebooks/search/uncased_L-12_H-768_A-12 -cpu

and calling the server via:

from bert_serving.client import BertClient
bc = BertClient(ip='localhost')

When I execute the line above in the Python console, it hangs forever (I even left it overnight). It doesn't raise any error, but I'm unable to proceed with the encoding because this line seems to be executing forever. Any idea what might be going wrong?

For completeness, I report the output I get from the bash command to start the server

2021-04-01 15:37:33.030106: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'cudart64_100.dll'; dlerror: cudart64_100.dll not found
2021-04-01 15:37:33.030590: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
usage: D:\Users\andrea.sottana\Desktop\notebooks\search\venv37\Scripts\bert-serving-start -model_dir=D:/Users/andrea.sottana/Desktop/notebooks/search/uncased_L-12_H-768_A-12 -cpu
                 ARG   VALUE
__________________________________________________
           ckpt_name = bert_model.ckpt
         config_name = bert_config.json
                cors = *
                 cpu = True
          device_map = []
       do_lower_case = True
  fixed_embed_length = False
                fp16 = False
 gpu_memory_fraction = 0.5
       graph_tmp_dir = None
    http_max_connect = 10
           http_port = None
        mask_cls_sep = False
      max_batch_size = 256
         max_seq_len = 25
           model_dir = D:/Users/andrea.sottana/Desktop/notebooks/search/uncased_L-12_H-768_A-12
no_position_embeddings = False
    no_special_token = False
          num_worker = 1
       pooling_layer = [-2]
    pooling_strategy = REDUCE_MEAN
                port = 5555
            port_out = 5556
       prefetch_size = 10
 priority_batch_size = 16
show_tokens_to_client = False
     tuned_model_dir = None
             verbose = False
                 xla = False

I:VENTILATOR:freeze, optimize and export graph, could take a while...
2021-04-01 15:37:53.088001: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'cudart64_100.dll'; dlerror: cudart64_100.dll not found
2021-04-01 15:37:53.088347: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
WARNING:tensorflow:From d:\users\andrea.sottana\desktop\notebooks\search\venv37\lib\site-packages\bert_serving\server\helper.py:186: The name tf.logging.set_verbosity is deprecated. Please use tf.compat.v1.logging.set_verbosity instead.

WARNING:tensorflow:From d:\users\andrea.sottana\desktop\notebooks\search\venv37\lib\site-packages\bert_serving\server\helper.py:186: The name tf.logging.ERROR is deprecated. Please use tf.compat.v1.logging.ERROR instead.

I:GRAPHOPT:model config: D:/Users/andrea.sottana/Desktop/notebooks/search/uncased_L-12_H-768_A-12\bert_config.json
I:GRAPHOPT:checkpoint: D:/Users/andrea.sottana/Desktop/notebooks/search/uncased_L-12_H-768_A-12\bert_model.ckpt
I:GRAPHOPT:build graph...
2021-04-01 15:38:04.980407: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
2021-04-01 15:38:04.988338: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'nvcuda.dll'; dlerror: nvcuda.dll not found
2021-04-01 15:38:04.989767: E tensorflow/stream_executor/cuda/cuda_driver.cc:318] failed call to cuInit: UNKNOWN ERROR (303)
2021-04-01 15:38:05.034880: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:169] retrieving CUDA diagnostic information for host: WSAMZN-TQ7TTEEQ
2021-04-01 15:38:05.035582: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:176] hostname: WSAMZN-TQ7TTEEQ
I:GRAPHOPT:load parameters from checkpoint...
I:GRAPHOPT:optimize...
I:GRAPHOPT:freeze...
I:GRAPHOPT:write graph to a tmp file: D:\Users\andrea.sottana\AppData\Local\Temp\1\tmpdjh442wv
I:VENTILATOR:optimized graph is stored at: D:\Users\andrea.sottana\AppData\Local\Temp\1\tmpdjh442wv
I:VENTILATOR:bind all sockets

...

[ilham-bintang]

Hi, probably this caused by wrong IP (in this case 'localhost'), so try remove the localhost and call via BertClient() instead.

If this occurs, kindly change the IP to 127.0.0.1

[pradeepdev-1995]

@ilham-bintang For me also the given code snippet is seems like hanging.

from bert_serving.client import BertClient bc = BertClient(ip='127.0.0.1')

The versions I am using are bert-serving-client==1.10.0 bert-serving-server==1.10.0. And running the code on ubuntu cpu machine.

Please help to solve this

[ilham-bintang]

Hi @pradeepdev-1995

Can you show me the output from your bert server? The IP address and port should be shown there

[pradeepdev-1995]

@ilham-bintang Here is the output of the bert server running console.

[ilham-bintang]

I mean the head part that show the config

[pradeepdev-1995]

@ilham-bintang Hope this is what you looking for

[pradeepdev-1995]

@ilham-bintang Its worked for me

I put the changes like this

from bert_serving.client import BertClient bc = BertClient(ip='127.0.0.1',port=5555)

Initially the port number I have given was incorrect , that's why it hanged . Now working fine.