Is import path supported in drpc gen?

[cindyzqtnew]

Hi guys,

I am trying to replace gRPC with dRPC in my testing project.

There are a few proto files in multiple directories and under the same directory "pkg", and some basic protos like "github.com.google/protobuf" under "vendor" directory. When I try to generate the pb.go files, it keeps returning the error:

protoc-gen-go-drpc: invalid Go import path "descriptor" for "google/protobuf/descriptor.proto"
The import path must contain at least one period ('.') or forward slash ('/') character.
See https://developers.google.com/protocol-buffers/docs/reference/go-generated#package for more information.

Here is my gen cmd:

for dir in ./pkg/abc/ ./pkg/efg; do \
  build/werror.sh /go_path/native/x86_64-pc-linux-gnu/protobuf/protoc
-Ipkg:./vendor/github.com:./vendor/github.com/gogo/protobuf:./vendor/github.com/gogo/protobuf/protobuf:./vendor/go.etcd.io:./vendor/github.com/grpc-ecosystem/grpc-gateway/third_party/googleapis 
--go-drpc_out=paths=source_relative,protolib=github.com/gogo/protobuf $dir/*.proto;

There indeed a file "./vendor/github.com/gogo/protobuf/protobuf/google/protobuf/descriptor.proto" with "option go_package = descriptor". But it's pulled automatically and already exists in my local repo for a long time. I doubt the option "-I" not work with drpc-gen, so it try to locate the basic gogo proto from a wrong location?

I've beed debugging for this issue for more than a week and look up all docs I could find but got no help.

Appreciate for any help!

[zeebo]

That is indeed interesting. I'll see if I can reproduce it locally with the details you have provided.

[zeebo]

I reproduced it locally. I think the problem is that gogo protobuf has an older descriptor.proto. Specifically, compare this line the current upstream has:

https://github.com/protocolbuffers/protobuf/blob/2a2a9b6e64e0cda49b0e5377cad58c790087e1c9/src/google/protobuf/descriptor.proto#L44

to this line in gogo:

https://github.com/gogo/protobuf/blob/226206f39bd7276e88ec684ea0028c18ec2c91ae/protobuf/google/protobuf/descriptor.proto#L44

I think this problem started because the protoc-gen-go-drpc command uses the google.golang.org/protobuf module at v1.27.1, and since v1.26.0, that module has more strict requirements for the go_package option (see https://github.com/protocolbuffers/protobuf-go/releases/tag/v1.26.0).

You can work around this by either modifying your include paths to have it find the updated descriptor.proto first, or you can add an option to the --go-drpc_out flag that specifies the import path like this: Mgoogle/protobuf/descriptor.proto=google.golang.org/protobuf/types/descriptorpb. That came from the documentation at https://developers.google.com/protocol-buffers/docs/reference/go-generated#package.

[cindyzqtnew]

I reproduced it locally. I think the problem is that gogo protobuf has an older descriptor.proto. Specifically, compare this line the current upstream has:

https://github.com/protocolbuffers/protobuf/blob/2a2a9b6e64e0cda49b0e5377cad58c790087e1c9/src/google/protobuf/descriptor.proto#L44

to this line in gogo:

https://github.com/gogo/protobuf/blob/226206f39bd7276e88ec684ea0028c18ec2c91ae/protobuf/google/protobuf/descriptor.proto#L44

I think this problem started because the protoc-gen-go-drpc command uses the google.golang.org/protobuf module at v1.27.1, and since v1.26.0, that module has more strict requirements for the go_package option (see https://github.com/protocolbuffers/protobuf-go/releases/tag/v1.26.0).

You can work around this by either modifying your include paths to have it find the updated descriptor.proto first, or you can add an option to the --go-drpc_out flag that specifies the import path like this: Mgoogle/protobuf/descriptor.proto=google.golang.org/protobuf/types/descriptorpb. That came from the documentation at https://developers.google.com/protocol-buffers/docs/reference/go-generated#package.

Since the protobuf version in my project is a bit old, upgrading to the latest version may introduce some risk. I'm thinking to modify the protobuf files by fixing the go_package issue and put them in another location (say "./vendor/github.com/golang/protobuf/protoc-gen-go/descriptor" and try to gen by adding "Mgoogle/protobuf/descriptor.proto=github.com/golang/protobuf/protoc-gen-go/descriptor":

for dir in ./pkg/abc/ ./pkg/efg; do \
  build/werror.sh /go_path/native/x86_64-pc-linux-gnu/protobuf/protoc
-Ipkg:./vendor/github.com:./vendor/github.com/gogo/protobuf:./vendor/github.com/gogo/protobuf/protobuf:./vendor/go.etcd.io:./vendor/github.com/grpc-ecosystem/grpc-gateway/third_party/googleapis 
--go-drpc_out=paths=source_relative,protolib=github.com/gogo/protobuf,Mgoogle/protobuf/descriptor.proto=github.com/golang/protobuf/protoc-gen-go/descriptor $dir/*.proto;

But it still returns the same error.

protoc-gen-go-drpc: invalid Go import path "descriptor" for "google/protobuf/descriptor.proto"
The import path must contain at least one period ('.') or forward slash ('/') character.

It looks like the "--go-drpc_out=M" not working...

[zeebo]

Sorry, I don't know why that's not working for you. I confirmed locally that it worked for me. Protoc invocations are very arcane, so I'd try double checking everything.