BricksLLM icon indicating copy to clipboard operation
BricksLLM copied to clipboard

Published 20 hours ago • verified publisher icon bricks-cloud

Fault transfer and load balancing

Open guleng opened this issue 1 year ago • 0 comments

Can the failover feature only be supported by OpenAI and Azure? Can't we support VLLM suppliers? Is there a load balancing function between two identical LLMs? So, how should I configure it? I can't find the configuration method

guleng avatar Jul 18 '24 07:07 guleng