ray
ray copied to clipboard
[Serve] design and implement transparent HTTP2 proxy
Currently we are using uvicorn to implement http proxy. We need to find an HTTP2 compatible server to to replace it and implement http and grpc proxy on top of it.
Maybe try hypercorn
This will allow us to not relying on gRPC proxy's server implementation and transparently passing payload between the replica and the proxy.