Changhyun Lee

@bojiang Sure, here are the additional information. OS: Debian GNU/Linux 12 Kernel version: 5.10.109-1.20220408.el7.x86_64

@bojiang , I have conducted the test about concurrent situation, but got similar result. 1. Prepare 4 pods running on different nodes. 2. Run `python3 bento_request_en.py` in each pods. 3....

@frostming > The balancing result is not correct since you are sending requests in sequence I ran the experiment because I expected the requests to be distributed across multiple workers,...

@frostming Click to expand (client code) ```python import numpy as np import requests import time import threading URL = "http://localhost:8080/classify" def classify_input_ids(): input_ids = np.random.randint(0, 100, (1, 16)).tolist() resp =...