grpc-web
grpc-web copied to clipboard
grpc
Bidi streaming use cases for grpc-web
As I understand it you're interested in possible use cases for streaming? Here's one (but suggestions for better alternative implementations are welcome).
Use case: Returning pages in a multipage view (e.g. list of threads in forum software)
In this use case the front-end JS opens a stream and sends a request (e.g. GetThreads(forum="General", page=0)) and gets a response containing only one page of results. On the server side, a cursor into the database query results (e.g. using PostgreSQL) is kept open, and on future requests in the stream (e.g. GetThreads(forum="General", page=1)) this cursor is used to get the results. This ensures a consist view of data (e.g. you don't see the same thread on page 2 as you just saw on page 1). When the stream is closed, the database cursor is closed. Without streams you would need to add logic about when to keep open or close database cursors, and under load balancing you need to find a way to make sure requests from the same client end up at the same server.
With the grpc-gateway wrapped in a websocket proxy (https://github.com/tmc/grpc-websocket-proxy) it's currently possible to do "bidi streaming" from the browser. Is using websockets for bidirectional streaming completely out of the question for grpc-web? I realise it would mean that any grpc-server would need to monitor for websocket connections, but maybe it could be a stepping stone before full trailer support is in the browsers?
@johanbrandhorst
Thanks for the point.
We are in the process of launching grpc-web as an alpha release, and will publish the plan on bidi support (road-map) after Q2.
@wenbozhu Do the grpc-web now support bidi streaming in the example? I saw the proto has defined bidi streaming API
rpc FullDuplexEcho(stream EchoRequest) returns (stream EchoResponse);
but the doc of Known gaps / issues says
No client-streaming or bidi-streaming support
One of our use case would be in the search bar, customer will enter the keyword sequentially and the search request/response will be implemented via bidi streaming
@HTChang no, the grpc-web repo version doesn't support bidi streaming. We probably should add a comment to the proto too to avoid confusion.
@stanley-cheung
@wenbozhu Have there been any developments on adding bidi support to the gRPC-Web spec?
We don't yet have a future-proof transport story that allows us to provide cross-browser support and at same time serve both OSS and internal users. Will publish the road-map doc soon, and we are very close to making the repo public.
@wenbozhu I'm sure you've heard people scream "but websockets!" at you a million times. But I'm genuinely curious, can you explain or link to an article or something that explains why Websockets shouldn't be used here?
Thanks @johanbrandhorst. Based on the comments on that issue (and the presence of a vote item in the ROADMAP.md), it sounds like it's still a possibility. Guess we'll just need to wait until the roadmap is updated with a decision.
Thanks for reading the ROADMAP.
This slide deck talked about websockets v.s. pure HTTP, with some stats from chrome.
We understand the desire to design a single API that works for both micro-service and Internet clients, among a few other use cases that could truly benefit from full-duplex streaming.
Hello, I want to write about my use case of bidi streaming. We are in the process of writing decentralized database. So, all data is encrypted and client must follow the b-tree with encrypted keys step-by-step to decrypt each key and say in which direction should be next step. We had good start in implementation this logic with grpc, but browser implementation without some wrappers (websocket, etc) of client is difficult now without bidi support. Our goal is the ability to use a decentralized database via browser having only a private key available.
Note that Imorobables grpc-web implementation has a beta bidi implementation over websockets. https://github.com/improbable-eng/grpc-web.
On Wed, Mar 28, 2018, 20:37 Dima [email protected] wrote:
Hello, I want to write about my use case of bidi streaming. We are in the process of writing decentralized database. So, all data is encrypted and client must follow the b-tree with encrypted keys step-by-step to decrypt each key and say in which direction should be next step. We had good start in implementation this logic with grpc, but browser implementation without some wrappers (websocket, etc) of client is difficult now without bidi support. Our goal is the ability to use a decentralized database via browser having only a private key available.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/grpc/grpc-web/issues/24#issuecomment-377009614, or mute the thread https://github.com/notifications/unsubscribe-auth/AGTFd-ucgkhhsClxjZPA0ogZ-AcX6fwNks5ti-ZhgaJpZM4K6Cd6 .
@johanbrandhorst the improbable beta bidi version is really premature. It is not documented and has no example code. All it does is just exporting a transport function with websocket, but doesn't really wire it up with grpc. If I miss something (i.e. branch) there, could you help point me to the right place of code? Thanks!
https://github.com/improbable-eng/grpc-web/blob/709b592779c35e17f581f9bf866f4cac4b0b93fb/ts/src/transports/Transport.ts#L64
@shanshanzhu the websocket proxy implementation is good. It wraps the Go http.ResponseWriter and io.Reader interfaces with websocket read/writes and passes the request to the gRPC Go implementation, so the wire protocol is pure gRPC-over-http2-over-websockets. Take a look at the implementation, it's beautiful :heart_eyes: https://github.com/improbable-eng/grpc-web/blob/72eb701d6f320ca324b3347c7925a720b553eae5/go/grpcweb/wrapper.go#L133, https://github.com/improbable-eng/grpc-web/blob/709b592779c35e17f581f9bf866f4cac4b0b93fb/ts/src/transports/websocket.ts#L51.
Furthermore, as I already pointed out in https://github.com/improbable-eng/grpc-web/issues/205, you can look at the tests for an example of how to use it: https://github.com/improbable-eng/grpc-web/blob/master/test/ts/src/client.websocket.spec.ts#L77.
The GopherJS gRPC-Web bindings have been using the websocket proxy on https://grpcweb.jbrandhorst.com since the start of the year.
Having said all of this, I think it is probably right to wait for Fetch API and Streams API to land in browsers before implementing this in this client.
Re: websockets, we would welcome all the gateways that provide inter-operability with gRPC end-points.
WS over HTTP/2 is being finalized and will be available in Chrome. We will evaluate if it makes sense to make WS a transport for grpc-web (but the bar to deploy this for Google's web apps will be high) and there will be details such as ping-pong, half-close etc to spec out ...
@johanbrandhorst does this mean there is a client side grpc-http2 implementation thats reading everything from the web sockets?
@wenbozhu My use case for bidirectional streaming is for an online game where the client is sending message based on what the user is doing and the server is sending messages to clients to broadcast all events. I could just do this over plain web sockets with protobuf but it'd definitely be nice to see support for this. Can we implement something similar to what @johanbrandhorst noted in improbable's grpc-web repo? It seems like it'd be really simple given its just a wrapper around the protocol and we could document that bidirectional streaming is done over web sockets. Later on, with WS over HTTP/2, we could enable support for that and then later for the streams API. This approach would ensure broad browser compatibility.
I don't think we should implement this in this repo until we have request streaming in Fetch. There is a third party, non-spec implementation that users can use if they need bidi streaming before then.
@johanbrandhorst the issue with that is Fetch will only work on bleeding edge browsers. I think its a pragmatic idea to include WebSockets over HTTP/1.1 as a fallback.
I disagree. This will by necessity be bleeding edge tech. The spec talks about eventually speaking the HTTP2 protocol, which is not supported by todays browsers. I don't think the aim of this project is to provide backwards compatibility to old browsers.
grpc-web does intend to support important browser use cases, i.e. features, versions, environments etc ... and the current spec is implementation details but important.
I don't necessarily agree tunneling grpc/http2 over websockets is a good or sane idea, and you are going to run into issues such as flow-control, half-close etc, to name a few.
One of the problems with websockets is that connectivity is not guaranteed (e.g. to support gmail) .. which means you always need a fallback. We don't yet want this level of complexity for grpc-web.
https://github.com/HTTPWorkshop/workshop2017/blob/master/talks/websockets.pdf
===
This bug is intended to collect use cases (title updated :) ... e.g. the case @nhooyr mentioned, could the API be actually better (e.g. resilient to failures) if client uses unary calls to post and uses server-stream to receive messages?
I don't necessarily agree tunneling grpc/http2 over websockets is a good or sane idea, and you are going to run into issues such as flow-control, half-close etc, to name a few.
Couldn't flow control be implemented at the grpc-web level? Likewise with half close?
One of the problems with websockets is that connectivity is not guaranteed (e.g. to support gmail) .. which means you always need a fallback. We don't yet want this level of complexity for grpc-web.
In what way is the connectivity of WebSockets not guaranteed?
e.g. the case @nhooyr mentioned, could the API be actually better (e.g. resilient to failures) if client uses unary calls to post and uses server-stream to receive messages?
The issue with that would be the constant authentication checks on every single message (of which there would be a lot) when doing an RPC. A bidirectional stream completely avoids this by only authenticating the connection at the start. I guess I could give each client a token before they start a game and then they pass that token on each request and we just directly compare that token instead of hashing for performance reasons on each RPC but still its more complicated than just a bidi stream.
Furthermore, while unary RPC+server stream avoids depending on the client side streaming connection remaining open, its not really more resilient because it'd be very easy to just reopen the bidi stream and we still have to worry about the server stream closing anyway. I don't think its more resilient in my case.
@nhooyr
Agreed, that performance win is a good case for bidi streaming.
Re: websocket connection failures, please check the doc on chrome stats. We don't really understand all the root causes, e.g. virus detection software etc.
Re: wire semantics, yes, grpc-web specific concepts need be introduced than just tunneling grpc/http2 over websockets (as a TCP-like bytestream). This to me adds unnecessary complexities than doing a grpc-web specific mapping to websockets (as a transport).
Re: websocket connection failures, please check the doc on chrome stats. We don't really understand all the root causes, e.g. virus detection software etc.
If that happens, there should be no fallback. Just throw an error. Its unfortunate but welp. I think in the vast majority of cases, web sockets will work fine and be a great fallback later for HTTP/2 WebSockets.
Re: wire semantics, yes, grpc-web specific concepts need be introduced than just tunneling grpc/http2 over websockets (as a TCP-like bytestream). This to me adds unnecessary complexities than doing a grpc-web specific mapping to websockets (as a transport).
Wouldn't grpc/http2 over web sockets take care of half close and flow control?
grpc/http2 only passes the payload over websockets, not http/2 itself .. but if the latter is the case, I really don't know how to reason about this.
In any case, I'd agree that ws over http2 will make ws a viable transport for grpc-web (only).
grpc/http2 only passes the payload over websockets, not http/2 itself .. but if the latter is the case, I really don't know how to reason about this.
Ah that sounds fine to me. I see what you mean now. We'd need a small wrapper protocol around websockets as well.