FR: Add tonic/prost rules

[UebelAndre]

This is a fairly common request. It would be nice if there were some tonic/prost rules in the @rules_rust//proto package since the existing rules there are using the protobuf-rs crate which only supports a (now) 3 year old version of protobuf (see https://github.com/stepancheg/rust-protobuf/issues/518).

[UebelAndre]

Right now there are two drafts for adding these rules:

• https://github.com/bazelbuild/rules_rust/pull/595
• https://github.com/bazelbuild/rules_rust/pull/479

As of right now, they're both pretty dated and likely will not work with the current rev of the rules. Hopefully this effort is revived 😄

[UebelAndre]

Hopefully, for folks looking to use a modern version of protobuf for proto generation, tonic/prost rules can satisfy this need since the existing proto rules do not (https://github.com/bazelbuild/rules_rust/issues/302 - blocked by https://github.com/stepancheg/rust-protobuf/issues/518).

[shikhar]

Even without tonic/prost rules, which would certainly make for less boilerplate, it's not too bad. https://github.com/shikhar/bazel-tonic is an example project I created that entirely uses rules_rust. codegen via https://github.com/shikhar/bazel-tonic/blob/main/src/protogen/BUILD

[GregBowyer]

I think it might be better to not have these in rules_rust but rather in rules_proto_grpc.

I have rules that can live in there in the works.

[UebelAndre]

I'd welcome it! I think it'd be better to keep proto/grpc rules all together. Even if they're for different languages.

[UebelAndre]

I saw https://github.com/rules-proto-grpc/rules_proto_grpc/issues/143 was recently opened. Seems like there's some interest in adding something like that to rules_proto_grpc.

[aaliddell]

Oh hi, I stubled across this from your cross-link :wave:

I'm stuck at the moment with Tonic/Prost support in rules_proto_grpc since they don't have protoc plugins available and instead require you use their own generation method that itself wraps protoc. It seems there were previously plugins and could be again in the future if there's demand. Otherwise I may have to figure out a way to wrap their generation method in a pseudo-plugin just to satisfy protoc, similar to what was done in those links above.

[GregBowyer]

Its not plugins that were previously available, its more we are all on the bleeding edge.

You are waiting on this https://github.com/tokio-rs/prost/pull/492.

I am going to try to push up my rules as they are for getting that working, there is some abuse of patch in cargo-raze to pull a few small tweaks to let things work.

I am going to look at what @Tuetuopay has done w.r.t a single lib.rs for this (its not needed fully for rules_gprc, but I think its a generally good change), along with some tweaks to see if descriptors can be included for grpc-reflection.

On Thu, Sep 23, 2021, at 4:11 PM, Adam Liddell wrote:

Oh hi, I stubled across this from your cross-link 👋

I'm stuck at the moment with Tonic/Prost support in rules_proto_grpc since they don't have protoc plugins available and instead require you use their own generation method that itself wraps protoc. It seems there were previously plugins and could be again in the future if there's demand. Otherwise I may have to figure out a way to wrap their generation method in a pseudo-plugin just to satisfy protoc, similar to what was done in those links above.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/bazelbuild/rules_rust/issues/915#issuecomment-926226455, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAA6X34UR4ZEHNIFHZ2IAC3UDOXZJANCNFSM5C7KA6TQ. Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[UebelAndre]

@GregBowyer is https://github.com/tokio-rs/prost/pull/492#issuecomment-966474758 meaningful info for you? If the changes there were to go in a separate crate, would it still be possible to use them to write prost/tonic rules?

[Sollimann]

I'm unable to use tonic_build with prost to generate stubs for my clients and serves when running my build.rs file. @shikhar https://github.com/shikhar/rules-rust-playground/tree/bazelify is this the solution to make it work? :thinking:

[Sollimann]

I have an alternative fix :muscle: :white_check_mark: It's not optimal, but this is a blocking issue for me, so I needed some way past it. First you do like you always would with cargo build / cargo build --release in a regular rust/cargo project. Normally when you build.rs file executes, the files go to the /target folder. Now, this didn't seem to work for me when using Bazel. So I figured that instead of having Bazel build the gRPC stubs and proto's, I'd rather do that with cargo, but I generate the stubs inside my /src folder instead of /target -folder. It works for me atleast.

cargo/
proto/
------ myproto.proto
src/
----- rust/
----------- mod.rs
----------- myproto.rs
----- lib.rs
BUILD.bazel
Cargo.toml
build.rs

build.rs

use std::fs;
use std::path::Path;
const PROTO_OUT_DIR: &str = "./src/rust";

fn main() -> Result<(), Box<dyn std::error::Error>> {
    if Path::new(PROTO_OUT_DIR).is_dir() {
        fs::remove_dir_all(PROTO_OUT_DIR)?;
    }

    fs::create_dir(PROTO_OUT_DIR).expect("Failed to create out dir");
    fs::write(
        format!("{}/mod.rs", PROTO_OUT_DIR),
        "pub mod sensors_service;",
    )?;

    let files = vec!["proto/sensors_service.proto"];
    let includes = vec!["proto"];

    tonic_build::configure()
        .build_client(true)
        .build_server(true)
        .format(true)
        .out_dir(PROTO_OUT_DIR)
        .compile(&files, &includes)?;
    Ok(())
}

Cargo.toml

[package]
name = "protos"
version = "0.1.0"
authors = ["sollimann <[email protected]>"]
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
tonic = { version = "0.4", features = ["prost"] }
prost = "0.7"
prost-types = "0.7"
serde = { version = "1.0", features = ["derive"] }


[build-dependencies]
tonic-build = "0.4"
prost-build = "0.7"

BUILD.bazel

load("@rules_rust//rust:defs.bzl", "rust_library")

package(default_visibility = ["//visibility:public"])

rust_library(
    name = "protos",
    srcs = glob(["**/*.rs"]),
    # crate_features = [
    #     "default",
    #     "std",
    # ],
    crate_root = "src/lib.rs",
    data = [],
    edition = "2021",
    rustc_flags = [
        "--cap-lints=allow",
    ],
    version = "0.1.0",
    visibility = [
        # had to add this line to be able to add /protos ass dep to grpc-connector-service
        "//services/grpc-connector-service:__subpackages__",
    ],
    deps = [
        "//protos/cargo:prost",
        "//protos/cargo:prost_build",
        "//protos/cargo:prost_types",
        "//protos/cargo:serde",
        "//protos/cargo:tonic",
        "//protos/cargo:tonic_build",
    ],
)

[UebelAndre]

https://github.com/tokio-rs/prost/pull/492#issuecomment-1079564170 seems like a relevant update to this thread

Rejoyce! I finally got time to port this, and I am happy to announce that I now have a protoc-gen-prost plugin that lives outside of this repository that is on par feature-wise with this PR! I'd even say that it is better: all options are supported, with a cleaner code, and better formatting than ever (it passes the generated code through prettyplease).

With the merge of #598 it will even not require any change to prost (although it still needs a git dep since it's unreleased yet). But as soon as this is released, the plugin can be pushed to crates.io. There are a few things that are left to do, mainly refacto a teeny bit to expose the internals as a lib, so that a future protoc-gen-tonic (my actual goal) can build on it. That'll be easy since I wrote the current code with this in mind.

Find it here: https://github.com/Tuetuopay/protoc-gen-prost

[Tuetuopay]

Popping in to let you know that I pushed protoc-gen-tonic, which can get handy.

[aidanalphafund]

Has anyone managed to get this to work with protoc-gen-tonic and/or protoc-gen-prost?

[titanous]

I've implemented support for protoc-gen-tonic and protoc-gen-prost in https://github.com/rules-proto-grpc/rules_proto_grpc/pull/202.