YongGangLee

I solved this problem by declaring CLIENT inside the function e.g. def predict_batchsize(inputs, model_name='building', batchsize=64, inp_desc=("INPUT__0", "FP32"), otp_desc=("OUTPUT__0", "FP32")): CLIENT = grpc_client.InferenceServerClient(url="192.168.128.29:8001") ... preds = CLIENT.infer(model_name=model_name, inputs=[inp], outputs=[otp]).as_numpy(otp_desc[0])