inference icon indicating copy to clipboard operation
inference copied to clipboard

Published 20 hours ago • verified publisher icon xorbitsai

ENH: Implement a load balancer

Open aresnow1 opened this issue 1 year ago • 2 comments

After #410 was merged, it should provide a load balancer to efficiently distributing requests.

aresnow1 avatar Sep 06 '23 07:09 aresnow1

require @qinxuye load balance for replica

yoke233 avatar May 13 '24 07:05 yoke233

This issue is stale because it has been open for 7 days with no activity.

github-actions[bot] avatar Aug 10 '24 19:08 github-actions[bot]

This issue was closed because it has been inactive for 5 days since being marked as stale.

github-actions[bot] avatar Aug 16 '24 19:08 github-actions[bot]