Suggestion to document gateway transcoding support for other languages.

[clehene]

If using the http annotations it's rather complicated to get code to compile across languages. Many resort to copying the common annotations in the project in order for protoc to generate the compile time dependencies (e.g. https://github.com/xolstice/protobuf-maven-plugin/issues/14) Instead you need https://github.com/googleapis/api-common-protos dependencies in each language to have them at compile time.

Also the protos need to be available to protoc when resolving the includes.

[johanbrandhorst]

Sounds great! Are you interested in providing some documentation in this area? Or an example repo perhaps?

[clehene]

@johanbrandhorst sure, I'm currently wrapping up the mess I've made while trying to get thing to work. Can you assign it to me, and I'll send a PR

[johanbrandhorst]

Thanks a lot!

[clehene]

@johanbrandhorst I've previously lost context on this, and it's a bit intricate. I'm finally back to this as I'm trying to figure out how to make it work for JS so got some context back, hence I'll braindump here before I lose context again :)

I'lll post what I have for Java and C# here.

When using annotations like

  rpc Echo(EchoRequest) returns (EchoResponse) {
    option (google.api.http) = { get: "/echo" };
  }

protoc needs the annotations.proto available at compile time.

annotations.proto typically comes from https://github.com/googleapis/googleapis/blob/master/google/api/annotations.proto - which is where grpc-gateway vendors them from.

Note that these are also avaialble in https://github.com/googleapis/api-common-protos/blob/master/google/api/annotations.proto

NOTE / TODO perhaps annotations.proto should instead be included in grpc binaries?

Once you generate code with protoc if you want to use it you need to resolve the actual language specific dependency for annotations.

https://github.com/googleapis/api-common-protos also has / provides language specific packages which will include Annotations - see https://github.com/googleapis/api-common-protos#packages

So for java there's https://mvnrepository.com/artifact/com.google.api.grpc/proto-google-common-protos For C# https://www.nuget.org/packages/Google.Api.CommonProtos/ Etc.

E.g. for C#

    <ItemGroup>
        <!--Common protos used for annotations.proto used for http transcoding-->
        <PackageReference Include="Google.Api.CommonProtos" Version="1.4.0" />
    </ItemGroup>

For Java / Maven

  <dependencies>
    <dependency>
      <groupId>com.google.api.grpc</groupId>
      <artifactId>proto-google-common-protos</artifactId>
    </dependency>
  </dependencies>

As these packages include the actual proto files, you'll likely be able to use any language specific toolchain for protoc. E.g. I'm using maven with xolstice so at generation time, protoc will need annotations.proto (E.g.) This allows me to build all language bindings with Maven. However in order to use the generated code you'll have to add the specific language dependency

For JS, I believe there's no package that contains google/api/annotations_pb.js The refrenced npm package from googleapis/api-common-protos - https://www.npmjs.com/package/google-proto-files only contains the .proto files. I did a google search and was only able to find the js in ~5 places (https://www.google.com/search?q="annotations_pb.js") Also note protocolbuffers/protobuf-javascript#62

I'll raise an issue on https://github.com/googleapis/api-common-protos and see where it gets me.

[johanbrandhorst]

Note that the reason there isn't a pre-generated version of annotations.proto for JS is probably because you can use JS in "dynamic" mode where it generates files as it needs them, and so only needs access to the proto files at runtime, not all the generated files.

[clehene]

@johanbrandhorst you're referring to http://dcode.io/protobuf.js/ ? TBH it's a bit confusing as it seems that protobuf and grpc have been going a different route of actually generating the JS https://developers.google.com/protocol-buffers/docs/reference/javascript-generated https://github.com/grpc/grpc-web

Also we want to support both direct grpc (grpc-web style) and transcoding.

As a side note - as non JS developer, already partially confused about the whole packaging jungle, I'm even more confused what's the right way to generate the JS SDK for a bunch of proto APIs and then use them. E.g. I have a public API and an internal one that depends on the public one. Normally they are both packaged and the internal libs depend on the public libs, but it's unclear what's the right way to generate some JS from proto and then package them and then use the resulting package as a dependency for the generation of a downstream SDK. I've been trying too figure out what others are doing and looked over googleapis, but the whole gapic / artman SDK generation is yet another beast.

[clehene]

It looks like it's possible to use decode.io protobuf.js with grpc-web https://github.com/grpc/grpc-web/issues/80

[johanbrandhorst]

No, even the Google implementation can be used dynamically AFAIK? In any case, the easiest way to work this out is to just implement it and test it. This is probably something we want to do anyway if we're writing documentation.

As another idea, maybe it's easier to generate the JavaScript parts from the generated swagger definitions?

[johanbrandhorst]

Also, while you can use this gateway together with the gRPC-Web proxy, you'd normally choose one or the other depending on the needs of your client(s).

[clehene]

Indeed the example there was for the improbable grpc-web (the whole protobuf.js vs protobuf/js grpc-web vs grpc-web is confusing at least)

Also, while you can use this gateway together with the gRPC-Web proxy, you'd normally choose one or the other depending on the needs of your client(s).

Besides allowing both gRPC and REST for services, with grpc-web you'd still need some way to perform health checks from load balancers.

[glerchundi]

@clehene we took another completely different approach to solve this issue. Make a proto to proto shrinker which removes any extension/annotation before passing it to the protoc generator.

[MalikWaseemJaved]

Found this thread useful, adding comment for future reference