What's the status of the project ?

[ngaut]

Excellent work! @stepancheg

What's the status of the project ? We want to use gRPC in our project TiKV, and we can help you to test the project.

[stepancheg]

Thanks for the feedback!

Project is is not ready to use yet. It can do simple requests, but

• HTTP/2 impementation is very incomplete, in particular WINDOW is not implementeed yet, that seems to be the reason behind bug #13
• performance optimizations have not beed started yet

I'm going to continue developing it soon, but I think you need to wait a couple of weeks to start evaluating it.

[ngaut]

Good to hear that. Thank you for listing the details! Please ping me if it's ready for testing. We will help you to test it as soon as possible.

[nstott]

I'm also looking forward to this.
:+1:

[calebmer]

Doing a great job, looking forward to this being ready 👍

Is it usable for development now and perhaps production in a few weeks?

[stepancheg]

Status update:

I've implemented "in" window in server. Missing parts are:

• respect client window in server (out window)
• implement window in rust client (in and out)

So, I HTTP/2 implementation should be feature complete soon, but honestly I doubt it will be ready for production. Especially if you need high performance.

(gRPC part is easy compared to HTTP/2 implementation).

Good news is that I've started implementing tests which verify rust implementation against go: https://github.com/stepancheg/grpc-rust/tree/master/long-tests It already helped to catch bugs.

[ngaut]

Great news! Well done, @stepancheg

[kanekv]

Is it threaded or evented? Do you use mio?

[stepancheg]

@Kane-Sendgrid it is evented. It uses tokio-core which in turn uses mio.

[stepancheg]

"In" window in client is also implemented now.

Now working on "out" window in client and server.

[siddontang]

Hi @stepancheg

Thanks for your great work first.

We decide to use gRPC in TiKV in production, see https://github.com/pingcap/tikv/issues/1424. But I still want to know what is the status of the project now? What do we need to pay attention to?

Btw, we want to use gRPC and Restful in one port like etcd, see https://github.com/grpc-ecosystem/grpc-gateway, https://coreos.com/blog/gRPC-protobufs-swagger.html, how can we do this in rust?

[stepancheg]

WINDOW_UPDATE is implemented now.

So, features of gRPC and HTTP/2 are implemented, although implementation still contains a lot of undiscovered bugs, and performance needs to be improved.

[kkimdev]

btw, have you talked to gRPC team? Maybe unlikely in the near future, but I wish this is officially maintained by gRPC someday.

[kkimdev]

Also, have you considered wrapping grpc.h instead of directly implementing in Rust http://www.grpc.io/grpc/core/grpc_8h.html ?

[cristicbz]

@stepancheg We're seriously evaluating a move from JSON/HTTP to grpc at re:infer. Our existing production code is Rust & Python, so we would love to help out with the Rust implementation.

I understand your time is limited, and I don't want to put any additional pressure on it, but if you could write up some kind of roadmap and TODOs, we'd be more super happy to contribute. Right now it's pretty hard for us to see what the holes in the implementation are (e.g. I noticed you recently moved to solicit for HTTP/2.0, is that move completed?; what is the status of load balancing and resolution? etc.) Alternatively, is there some IRC channel you hang out on where we could chat at some point?

Thanks so much for all your work (on this and rust-protobuf which we're already using for storage)!

[stepancheg]

@kkimdev

Also, have you considered wrapping grpc.h instead of directly implementing in Rust http://www.grpc.io/grpc/core/grpc_8h.html ?

Yes, I've considered it, but I decided against it, because:

• it is easier for users to have pure rust implementation than requiring from them to compile C implementation
• implementing HTTP/2 from scratch seems to me marginally harder than implementing correct bindings to C implementation. Probably I'm wrong, but that is how it looked to me when I started the implementation. There was a "solicit" implementation of HTTP/2 which I was hoping to use. It turned out that solicit cannot be used directly for async implementation, and I had to copy and rewrite large part of it, but as I said, I didn't expect that at start.
• eventually Rust community will have decent implementation of HTTP/2 which is the largest part of gRPC implementation, so there will be no need for C implementation

Anyway, if someone wishes to contribute bindings to C implementation to grpc-rust, I will be happy to accept it. I think it would be great I grpc-rust users could have an option to switch implementations preserving the same interface. I would do it myself, but unfortunately now I have no time for that.

[kkimdev]

Yeah, I would love to have libraries in pure Rust as much as possible. Though, my impression is that Google's grpc implementation is really a non-trivial work that it will be almost impossible for us to catch up the quality and the maintenance work. For example, they have continuous performance benchmarks and monitor regressions and improvement headrooms https://performance-dot-grpc-testing.appspot.com/explore?dashboard=5685265389584384 , and they put good efforts in optimizing their grpc C implementation https://github.com/grpc/grpc/projects/1. So personally, although it sucks to manage C/C++ compiling, I'd still prefer to use their grpc implementation for production use at least today.

I think an option to switch the underlying implementation is a great idea. Btw, Rust's bindgen is able to generate c-binding well, C++ binding needs some work though https://github.com/grpc/grpc/issues/9316 .

[stepancheg]

cristicbz@

if you could write up some kind of roadmap and TODOs, we'd be more super happy to contribute

I've updated the TODO list and created a could of issues on the github project. You are welcome to grab anything.

I noticed you recently moved to solicit for HTTP/2.0, is that move completed?

Actually it is the opposite. I used solicit and maintained a series of patches againtst it. Solicit project seems to be abandoned, and it is not designed for async processing, so I decided to import part of the project source code into grpc-rust. (I should mention original author somewhere).

what is the status of load balancing and resolution

I'm not sure what you mean, but probably nothing is done in that area.

Alternatively, is there some IRC channel you hang out on where we could chat at some point?

There's a channel on gitter: https://gitter.im/grpc-rust/Lobby for a quick chat.

[cristicbz]

@stepancheg Thanks for all of that, I won't have time to look at it for a little while, but just wanted to clarify

what is the status of load balancing and resolution

I'm not sure what you mean, but probably nothing is done in that area.

This is the stuff I'm talking about:

https://github.com/grpc/grpc/blob/master/doc/naming.md https://github.com/grpc/grpc/blob/master/doc/load-balancing.md

[stepancheg]

Created a separate issue for grpc.h, if anything will be done in that area, it should be discussed there.

[stepancheg]

An issue about load-balancing.

Not sure "naming" is important at this point of grpc-rust development, but if it is, please open an issue.

[mieubrisse]

Hey @stepancheg , wanted to resurrect this thread to ask the same question about the project status? The README says "It basically works, but not suitable for production use.", but I'd really prefer to use this project rather than prost because this project provides protoc plugins that are agnostic to build.rs

[stepancheg]

ask the same question about the project status?

Same as before. Unfortunately, I don't have much time to implement it properly. I tried to keep existing things working, but the amount of work need to do everything properly is huge.

[nathan-at-least]

Hi, I came to this project by way of rust-protobuf which I found because I wanted to build rust crates from .proto files with no external toolchain dependencies (ie protoc). I poked around and noticed https://github.com/stepancheg/grpc-rust/issues/139#issuecomment-438497711:

Pure rust grpc codegen is not yet implemented.

"Oh great!", I thought, "then it's on the roadmap". But after poking around, scanning issues, and looking at commit history, I can't tell how active this project is, so I have a few suggestions:

Update the README with development status. No shame in any status (not maintained, maintained with limited time/attention, etc…), and making it clear could help everyone who shows up and may prompt more contributors. Add a roadmap to the README, or use github project features to prioritize the backlog of issues. I would really appreciate a grpc tool that is (a) async and (b) does not rely on non-cargo toolchain (ie protoc), because it feels like that would be a boon to rust grpc support! Let us know how we can help and thanks for a great project.

[stepancheg]

I think at the moment the best way to proceed is to implement codec for tower-grpc if you want to use rust-protobuf. This project is far from being production ready, and mostly abandoned. That shouldn't be too hard.

Another option is to patch prost to use protobuf parser from rust-protobuf, that shouldn't be too hard as well, because rust-protobuf has all the machinery to emit data in protobuf format (as protoc does). Basically, protoc but as a library can be built from rust-protobuf.