Add support for gRPC

[jwulf]

Is it possible to write a gRPC client application in Deno?

[benjamingr]

Sure, what do you mean?

[ry]

There is no gRPC module yet.

[tuddman]

would this be the way to go? https://github.com/stepancheg/grpc-rust

[sleipnir]

Hello @tuddman , I understand that Deno is written over Rust but I think the question is about how I use gRPC in Deno, how do I write a gRPC client or server code in Deno? It's possible? Is there a library ready to use? If not, how to use the link you sent to provide a Javascript / Typescript api at the user level?

[ajile-in]

Looking forward to the possibility of using gRPC with Deno.

[sleipnir]

I believe that due to the popularity of gRPC it would be great to see support in the Deno standard library, like any other form of IO

[theoutlander]

I would like to help with this integration, but I'm wondering if the best option is integration with grpc-rust or writing a typescript library? What about porting something like https://github.com/grpc/grpc-web/tree/master/packages/grpc-web as a typescript lib?

[kitsonk]

IMO, this is something that should start external to Deno/std and move into std. It should either be a native module or Web Assembly.

[badsyntax]

It should be noted the current advised approach of using gRPC in Node.js is to use the pure JavaScript module which uses Node.js's built-in modules (http2) instead of a native module built in c++. The module is built in TypeScript and perhaps can be used for inspiration for something in Deno, assuming http2 support is provided by Deno at some point. cc @murgatroid99

[murgatroid99]

Hi. I'm the author of the existing gRPC libraries for Node.js. I think that porting @grpc/grpc-js to Deno might be the way to go, but that would depend on how close Deno's eventual HTTP/2 implementation is to Node's http2 module, semantically. Other built in modules may also be a concern, particularly tls.

I would not suggest porting grpc-web. That library implements a different protocol to account for the restrictions of browser APIs. It needs a proxy to talk to regular gRPC servers.

[sleipnir]

I aggre about grpc-web maybe this should be another Deno module

[yurianthedev]

@murgatroid99 I have a doubt, how is gRPC-node generating code from Proto right now?

[murgatroid99]

There are two major methods: one is to use the @grpc/proto-loader package to load .proto files and generate classes and object at runtime. That uses Protobuf.js internally.

The alternative is to use the protoc tool distributed with a plugin in the grpc-tools package to generate JavaScript files that depend on the google-protobuf package.

I want to note that neither of these is strictly necessary in general to use grpc. The gRPC protocol (and the library) can handle any kinds of serializable messages.

[yurianthedev]

Ok, let me understand a little bit. So first we need to implement something like grpc-deno, which should have support for any kind of serializable message. Then adapting (or build a new one) the tooling for generating grpc code that uses grpc-deno as a reference, from proto (or any other) to TypeScript/Javascript.

[murgatroid99]

That looks about right.

One thing I want to note is that there is a kind of generic interchange format for defining a collection of services that grpc.loadPackageDefinition accepts and @grpc/proto-loader generates, and there is also an option to have grpc-tools generate it. That interchange format is defined as PackageDefinition in this document. I think it would be good if any implementation for Deno can load the same kind of objects.

Ideally I think gRPC for Deno should have the same API and functionality as the existing @grpc/grpc-js, but I recognize that that may not be feasible.

[yurianthedev]

@murgatroid99 there is an article or a document explaining how grpc-node (or other C-grpc-based libraries) uses the core-grpc implementation?

[yurianthedev]

Another thing @murgatroid99, What do you think about building a grpc-deno on top of tonic (pure Rust implementation of grpc).

[murgatroid99]

I'm not aware of such a public document about using the gRPC core library. We did most of that work before we started doing our design work publicly. The simple answer, though, is that we defined a public API for JavaScript users (in this case), and then an intermediate API, that the implementation of the public API could consume and that we could reasonably implement in C++ using the core API. That intermediate API is considered non-public and not guaranteed to be stable. If you look at the grpc package directories, the ext/ directory contains the implementation of that intermediate API and the src/ directory contains the implementation of the public API.

The Node library in particular is unusual in that the gRPC core library has a libuv implementation of the low-level network interaction that was created specifically for the Node library, which can be enabled using a compiler flag. So the Node gRPC library enables that to simplify its interaction with the event loop.

I think it's important to mention that the native addon was a maintainability nightmare, to the point that we decided to reimplement the library purely in TypeScript as @grpc/grpc-js and we are phasing out the use of the native addon-based library in favor of the pure TypeScript implementation.

Regarding implementing this on top of tonic, I think the important question is how close the desired JavaScript API is to the tonic API, semantically. In addition, how much extra work will it be to maintain the Deno equivalent of a native addon using a Rust library? And how does the total implementation work there compare with modifying @grpc/grpc-js for use with Deno, once the necessary standard libraries are available.

[yurianthedev]

I have taken a look at your pure TS implementation of grpc, and you are using only a single node dependency, so a deno library could be hard based on that implementation right? Based on what you said about the core library could be a nightmare in the long term, even with a Rust implementation.

[theoutlander]

I like the typescript based approach. Maybe we need to implement http/2 first, then it is mostly straightforward.

On Sat, Aug 15, 2020 at 11:36 AM yurianxdev [email protected] wrote:

I have taken a look at your pure TS implementation of grpc, and you are using only a single node dependency, so a deno library couldn't be hard based on that implementation? Based on what you said about the core library could be a nightmare in the long term, even with a Rust implementation.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/denoland/deno/issues/3326#issuecomment-674433627, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAFW4HFVFN36VLGRWRRF75DSA3ITXANCNFSM4JMRSKMA .

[yurianthedev]

There is no progress on HTTP2 (#3995) so if we want to follow the pure TS approach we need to implement it. That will kill two birds with one shot, I like it. @murgatroid99 what do you think about a pure TS implementation based on yours?

[kitsonk]

HTTP2 will eventually land.

Again, this needs to start as something seperate from std or the Deno cli.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[jd1378]

any news on this ?

[anonrig]

Hi! I'm a maintainer from mali (GRPC microservice framework) and wanted to stay in the loop. We're currently investigating supporting Deno in Mali. If you're interested in this support, please send us a message from https://github.com/malijs/mali/issues/241

[brunoluiz]

Seems Deno 1.9, released today, implements HTTP 2.0 🎉

https://deno.com/blog/v1.9#native-http%2F2-web-server

[murgatroid99]

The post says that it supports full duplex communication. That's good. Does it also have full support for trailers, on the client side and the server side? That is important for a gRPC implementation.

[prohazko2]

Was playing around with gRPC implementation and almost succeed with basic unary calls serving https://github.com/prohazko2/deno-grpc/blob/0.1.3/serve.ts

1. http2 with Deno.serveHttp was ok
2. protobuf msgs decoding/encoding with protobuf.js was ok
3. thing that I didn't figure out - how to send OK status back to client

Similar node's @grpc/grpc-js response has two server trailers:

[0] Received server trailers:
		grpc-status: 0
		grpc-message: OK

[0] received status code 0 from server
[0] received status details string "OK" from server

Seems like they are sent with this call to node http2stream.sendTrailers @grpc/[email protected] ... server-call.ts

But I didn't have any luck with sending them with Deno's respondWith / Response API's

[bartlomieju]

@prohazko2 trailing headers are not currently supported. Could you open a new issue with your use case? We'll try to look into it soon.

[prohazko2]

@bartlomieju sure, here it goes https://github.com/denoland/deno/issues/10214

[prohazko2]

Hi, If anyone still interested - i made some progress on my lib https://github.com/prohazko2/deno-grpc It now can make and serve unary calls. Main bummer is - it doesn't use Deno's HTTP/2 bindings but uses JS impl from now deprecated https://github.com/molnarg/node-http2 repo. Also not expect much from it - I do it only for fun and have no intentions on implementing full gRPC spec

[badsyntax]

As per https://github.com/denoland/deno/issues/10214#issuecomment-821723392

I don't think we will be supporting them either if the web does not.

Does this effectively mean we cannot port grpc-js to deno?

[benatkin]

@badsyntax no, it means that it isn't going to be in core.

I think it might be implemented in an ideal fashion as an ffi feature, and that we might need an --allow-ffi allow list or something.

[stabai]

At the risk of beating a dead horse, there is an unobvious use case that Deno would be awesome for, but also depends on gRPC support. Plugins for Terraform, Aspect, and other tools that use this pattern can be written in any framework that supports gRPC. For developers who work with gRPC, this also means interoperability with all of their own tools/services in their development platform. The current necessity to do these things in Node or some other language is far from ideal, especially for those who are trying to adopt Deno.

[stabai]

@benatkin

I think it might be implemented in an ideal fashion as an ffi feature, and that we might need an --allow-ffi allow list or something.

Is there another issue tracking that for us to follow?

[kitsonk]

To give context, we believe that integrated gRPC to the Deno CLI is an important feature and are committing core team resources to delivering it.