firebook

期待加个类，正确的关闭client Socket 直接调用`java.net.Socket#close()`，会导致服务端报：`java.net.SocketException: Connection reset` 原因是close会先发FIN，再发RST，客户端没有等待服务端的FIN 相关链接：https://docs.oracle.com/javase/8/docs/technotes/guides/net/articles/connection_release.html

加权的负载均衡策略

1

` size_t size = service_list.size(); int begin_index = rand() % size; int rand_index = begin_index; do { ret = mnsc_thrift_client->CreateThriftClient(service_list[rand_index].ip, service_list[rand_index].port, timeout_); if (SUCCESS == ret) { return SUCCESS; }...

lost watch when client send two request without nonce

1

log ``` [2022-04-25 19:13:25.993] [DEBUG] [grpc-default-executor-0] [i.e.controlplane.server.OspDiscoveryServer] >>> [0] open stream from [2022-04-25 19:13:26.009] [DEBUG] [grpc-default-executor-0] [i.e.controlplane.server.OspDiscoveryServer] >>> [0] request type.googleapis.com/envoy.api.v2.ClusterLoadAssignment[r1] with nonce from version 3 [2022-04-25 19:13:26.684] [DEBUG] [grpc-default-executor-0]...

lose watch when SimpleCache respond return false in respondWithSpecificOrder

1

For ADS mode, class io.envoyproxy.controlplane.cache.SimpleCache will respond to client when setSnapshot. When respond(watch, snapshot, group); return false, respondWithSpecificOrder will lose the watch, since respond is not complete, client will not...

## Why do you need it? 4xx 错误不想要 fallback，5xx 错误才需要 fallback ## How could it be? AI 网关的 fallback，可以根据不同错误码做定制，提供配置页面，比如 4xx 的错误码不 fallback，5xx 的错误码 fallback ## Other related information Add any...

## Why do you need it? 如果首 token 响应过长，可能是后端模型服务处理不过来，我们需要及时中断请求，把请求 fallback 到另外一个模型服务 ## How could it be? 可以在网关界面提供超时配置，当首 token 响应时长命中超时，返回错误码 ## Other related information Add any other context or screenshots about...

[Bug] fix: error when parse complex content in completion body

1

## Pull Request Description AiBrix will raise an error when parse complex content in completion body error message: ``` /v1/chat/completions json unmarhsal valid messages with complex content%!(EXTRA string=/v1/chat/completions json unmarhsal...