Support single include file

[SephVelut]

tonic-build: 0.5.2

Trying to use googleapi. Google api proto files are in their own directory within the project projectname/protos. projectname/ projectname/protos projectname/src projectname/src/speech projectname/build.rs projectname/target etc

// build.rs
fn main() -> Result<(), Box<dyn std::error::Error>> {
    tonic_build::configure()
        .build_client(true)
        .build_server(false)
        .format(false)
        .out_dir("src/speech")
        .compile(&["protos/googleapis/google/cloud/speech/v1/cloud_speech.proto"], &["protos/googleapis"])?;

    Ok(())
}

cargo build and the .rs files are generated in src/speech. Now the problem.

// main.rs

#[path = "speech"]
pub mod speech {
   #[path = "google.rpc.rs"]
   pub mod rpc;

    #[path = "google.longrunning.rs"]
    pub mod longrunning;

    #[path = "google.cloud.speech.v1.rs"]
    pub mod transcriber;
}

error[E0433]: failed to resolve: there are too many leading `super` keywords
   --> src/speech/google.cloud.speech.v1.rs:588:53
    |
588 |     pub error: ::core::option::Option<super::super::super::rpc::Status>,
    |                                                     ^^^^^ there are too many leading `super` keywords

error[E0433]: failed to resolve: there are too many leading `super` keywords
   --> src/speech/google.cloud.speech.v1.rs:726:3457
    |
726 | ... :: Response < super :: super :: super :: super :: longrunning :: Operation > , tonic :: Status > { self . inner . ready () . await . ...
    |                                              ^^^^^ there are too many leading `super` keywords

error[E0698]: type inside `async fn` body must be known in this context
   --> src/speech/google.cloud.speech.v1.rs:726:3690
    |
726 | ... . into ())) }) ? ; let codec = tonic :: codec :: ProstCodec :: default () ; let path = http :: uri :: PathAndQuery :: from_static ("/...
    |                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ cannot infer type for type parameter `U`
    |
note: the type is part of the `async fn` body because of this `await`
   --> src/speech/google.cloud.speech.v1.rs:726:3847
    |
726 | ...peech/LongRunningRecognize") ; self . inner . unary (request . into_request () , path , codec) . await } # [doc = " Performs bidirecti...
    |                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error[E0698]: type inside `async fn` body must be known in this context
   --> src/speech/google.cloud.speech.v1.rs:726:3690
    |
726 | ...t codec = tonic :: codec :: ProstCodec :: default () ; let path = http :: uri :: PathAndQuery :: from_static ("/google.cloud.speech.v1...
    |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ cannot infer type for type parameter `M2` declared on the associated function `unary`
    |
note: the type is part of the `async fn` body because of this `await`
   --> src/speech/google.cloud.speech.v1.rs:726:3847
    |
726 | ...peech/LongRunningRecognize") ; self . inner . unary (request . into_request () , path , codec) . await } # [doc = " Performs bidirecti...
    |                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error[E0283]: type annotations needed
   --> src/speech/google.cloud.speech.v1.rs:583:28
    |
583 | #[derive(Clone, PartialEq, ::prost::Message)]
    |                            ^^^^^^^^^^^^^^^^ cannot infer type
    |
    = note: cannot satisfy `_: Default`
note: required by `std::default::Default::default`
   --> /home/gromneer/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/default.rs:116:5
    |
116 |     fn default() -> Self;
    |     ^^^^^^^^^^^^^^^^^^^^^
    = note: this error originates in the derive macro `::prost::Message` (in Nightly builds, run with -Z macro-backtrace for more info)

[DerekStrickland]

I have the same errors. It actually seems like at least some of them are coming from prost a layer lower. The first fix for me was that the mod hierarchy in rust has to match the directory structure in the source repository where the proto files are found. This snippet from my main.rs has me past the "too many super" error. Still debugging the "cannot infer type" stuff. I'll update if/when I figure out the correct config.

mod hashicorp {
    mod nomad {
        pub mod plugins {
            pub mod drivers {
                include!(concat!(env!("OUT_DIR"), "/hashicorp.nomad.plugins.drivers.proto.rs"));
            }
        }

        pub mod shared {
            pub mod hclspec {
                include!(concat!(env!("OUT_DIR"), "/hashicorp.nomad.plugins.shared.hclspec.rs"));
            }

            pub mod structs {
                include!(concat!(env!("OUT_DIR"), "/hashicorp.nomad.plugins.shared.structs.rs"));
            }
        }
    }
}

[LucioFranco]

There is a google protobuf example. I believe using the new feature from prost-build to generate a single include file should fix a lot of this. We can look into adding similar support into tonic.

[DerekStrickland]

It did for me 😸

[djc]

I'm running into similar issues. I've synced a subset of the googleapis repo into the proto dir in my crate and used the build.rs contents suggested in the documentation:

fn main() {
    tonic_build::configure()
        .build_server(false)
        .compile(
            &["proto/googleapis/google/cloud/secretmanager/v1/service.proto"],
            &["proto/googleapis"],
        ).unwrap();
}

However, this ends up with up to four super::super::super::super which rustc doesn't like (can't imagine why). What is "the new feature" from prost-build I should use to fix this? Would be willing to help out getting this fixed.

(I think this is the same thing as #767?)

[LucioFranco]

The feature should be supported on the latest tonic?

[LucioFranco]

https://docs.rs/tonic-build/0.6.0/tonic_build/struct.Builder.html#method.include_file

[djc]

Okay, yes, I got that to work. Not sure why this isn't/couldn't be the default? Also might be nice to extend the documentation to mention that you might want to use something like include!(concat!(env!("OUT_DIR"), "/proto.rs"));.

[kcking]

For others reading this issue looking for a workaround, the thing that worked for me specifically when consuming google cloud protos was to add .include_file("mod.rs") to the tonic-build invocation, and then change the include to tonic::include_proto!("mod")

[djc]

For what it's worth, I wrote some code for opentelemetry-stackdriver that uses an integration test to drive code generation and generate it into a neat module hierarchy inside you're crate's src dir. This avoids regenerating code in a build script (for libraries) and makes the generated code easily accessible to an IDE.

[suiluj]

For what it's worth, I wrote some code for opentelemetry-stackdriver that uses an integration test to drive code generation and generate it into a neat module hierarchy inside you're crate's src dir. This avoids regenerating code in a build script (for libraries) and makes the generated code easily accessible to an IDE.

@djc thank you very much for generating the opentelemetry-proto. I just did not get it to work for the opentelemetry trace collector stuff. Now I can just use your module.

It's my first rust project so perhaps that's why I do not get it to work even with the hints here. But I have the feeling that gRPC code generation is a lot easier when I'm developing in python or golang.