clip-as-service CPU seems to be bottleneck, as opposed to GPU

Prerequisites

Please fill in by replacing [ ] with [x].

[x] Are you running the latest bert-as-service?
[x] Did you follow the installation and the usage instructions in README.md?
[x] Did you check the FAQ list in README.md?
[x] Did you perform a cursory search on existing issues?

System information Ubuntu 18.04.4 Python 3.7.3 Tensorflow 1.15.0 GPU RTX2080 TI CPU i9-9900K (8 cores / 16 threads) RAM 64GB DDR4 CUDA 10.2 Driver Version 440.59

Some of this information can be collected via this script.

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): 18.04.4
TensorFlow installed from (source or binary): Don't remember
TensorFlow version: 1.15.0
Python version: 3.7.3
bert-as-service version: 1.10.0
GPU model and memory: RTX 2080 Ti 11GB
CPU model and memory: i9 9900K 64GB DDR4 RAM

Description

Please replace YOUR_SERVER_ARGS and YOUR_CLIENT_ARGS accordingly. You can also write your own description for reproducing the issue.

I'm using this command to start the server:

bert-serving-start -model_dir downloads/uncased_L-24_H-1024_A-16/ -num_worker=1 -max_batch_size 1024 -max_seq_len 8 -prefetch_size 512

and calling the server via:

bc = BertClient(ignore_all_checks=True)
bc.encode(list_of_strings)

Then this issue shows up: I am gathering about 8000 strings of length 8 ngrams and sending them to bert-serving server. I would expect the GPU utilization to increase and for the 8000 strings to be processed in a few seconds. Instead, my CPU utilization is capped out, and GPU utillization is practically 0, and it takes a few minutes to process. If I change the settings to run on my CPU, it takes significantly longer than this, so I am confident I am running on my GPU (also the server indicates it is running on my GPU). Am I doing something grossly wrong here? Thanks in advance! ...

Feb 18 '20 00:02 Atrus619

same issue on my server. I am using a ubuntu server with 4 gpus, and the tensorflow version is 1.15. My CPU utilization is capped out, and GPU utillization is practically 0.