gRPC Reflection doesn't support maps

[lynxtaa]

grpcurl errors if reflection is enabled and protobuf has map fields defined:

$ grpcurl -plaintext localhost:5000 helloworld.Greeter/SayHello

Error invoking method "helloworld.Greeter/SayHello": failed to query for service descriptor "helloworld.Greeter": proto:
message field "helloworld.HelloReply.field" is an invalid map: incorrect implicit map entry name

// helloworld.proto

syntax = "proto3";

package helloworld;

service Greeter {
  rpc SayHello (HelloRequest) returns (HelloReply) {}
}

message HelloRequest {
  string name = 1;
}

message HelloReply {
  map<string, string> field = 1;
}

// server.mjs

import {
    Server,
    loadPackageDefinition,
    ServerCredentials,
} from '@grpc/grpc-js';
import { loadSync } from '@grpc/proto-loader';
import reflection from '@grpc/reflection';

const PROTO_PATH = './helloworld.proto';

const server = new Server();
const packageDefinition = loadSync(PROTO_PATH);
const proto = loadPackageDefinition(packageDefinition);
new reflection.ReflectionService(packageDefinition).addToServer(server);

server.addService(proto.helloworld.Greeter.service, {
    sayHello: (call, callback) => {
        callback(null, { message: 'Hello' });
    },
});

server.bindAsync('localhost:5000', ServerCredentials.createInsecure(), () => {
    server.start();
});

In Postman and Insomnia reflection works fine. Removing map field also helps

[jhump]

@lynxtaa, I suspect the issue is in the server's reflection response.

You can test this from the command-line using grpcurl itself.

• First, you'll need to give gRPC the definition of the reflection service as a protobuf source file (since it doesn't like your server's reflection implementation). You can download that from here.
• Then run the following command, to verify that it can successfully return a descriptor for your service:

grpcurl -plaintext -proto reflection.proto \
    -d '{"file_containing_symbol": "helloworld.Greeter"}' \
    localhost:5000 \
    grpc.reflection.v1.ServerReflection/ServerReflectionInfo

• If that doesn't work, it may shed light on the issue and provide enough info in the error as to what's gone awry.
• If that does work, then you can use the following to actually extract/decode the descriptor. The first example shows how to do it using the buf command-line tool. The second example shows how to do it with protoc.

# with buf, emits JSON
grpcurl -plaintext -proto reflection.proto \
    -d '{"file_containing_symbol": "helloworld.Greeter"}' \
    localhost:5000 \
    grpc.reflection.v1.ServerReflection/ServerReflectionInfo \
| jq -r ".fileDescriptorResponse.fileDescriptorProto[0]" \
| base64 -D \
| buf convert buf.build/protocolbuffers/wellknowntypes \
    --type google.protobuf.FileDescriptorProto \
| jq . 

# with protoc, emits text format
grpcurl -plaintext -proto reflection.proto \
    -d '{"file_containing_symbol": "helloworld.Greeter"}' \
    localhost:5000 \
    grpc.reflection.v1.ServerReflection/ServerReflectionInfo \
| jq -r ".fileDescriptorResponse.fileDescriptorProto[0]" \
| base64 -D \
| protoc google/protobuf/descriptor.proto \
    --decode google.protobuf.FileDescriptorProto

If that works, can you drop the resulting JSON or text format into a reply? That would likely help identify the issue.

If that does not work, and the server is returning an "unimplemented" error code, it probably only supports v1alpha of the reflection service instead of v1. That should be fine (grpcurl should support both), so try the above again but this time using this version of reflection.proto instead and then replacing grpc.reflection.v1.ServerReflection/ServerReflectionInfo with grpc.reflection.v1alpha.ServerReflection/ServerReflectionInfo in the command

[lynxtaa]

Thanks for a quick response! I've tried the above command and here's my output using protoc:

$ grpcurl -plaintext -proto reflection.proto     -d '{"file_containing_symbol": "helloworld.Greeter"}'     localhost:5000     grpc.reflection.v1.ServerReflection/ServerReflectionInfo | jq -r ".fileDescriptorResponse.fileDescriptorProto[0]" | base64 -d | ../node_modules/grpc-tools/bin/protoc google/protobuf/descript
or.proto     --decode google.protobuf.FileDescriptorProto

name: "helloworld.proto"
package: "helloworld"
message_type {
  name: "HelloRequest"
  field {
    name: "name"
    number: 1
    label: LABEL_OPTIONAL
    type: TYPE_STRING
  }
}
message_type {
  name: "HelloReply"
  field {
    name: "field"
    number: 1
    label: LABEL_REPEATED
    type: TYPE_MESSAGE
    type_name: "Field"
  }
  nested_type {
    name: "Field"
    field {
      name: "key"
      number: 1
      label: LABEL_OPTIONAL
      type: TYPE_STRING
    }
    field {
      name: "value"
      number: 2
      label: LABEL_OPTIONAL
      type: TYPE_STRING
    }
    options {
      map_entry: true
    }
  }
}
service {
  name: "Greeter"
  method {
    name: "SayHello"
    input_type: ".helloworld.HelloRequest"
    output_type: ".helloworld.HelloReply"
  }
}
syntax: "proto3"

[jhump]

So I take it the library you are using parses proto sources, instead of using descriptors compiled by a tool like protoc or buf?

Because that descriptor is indeed wrong. Map types should synthesize messages with Entry as a name suffix. In this case, it should be named FieldEntry, not simply Field.

Here's the logic in the official Protobuf C++ implementation, which rejects this sort of descriptor (in particular, the check on lines 9067-9068): https://github.com/protocolbuffers/protobuf/blob/v31.1/src/google/protobuf/descriptor.cc#L9055-L9072

And here's where this actual message is coming from in the Protobuf Go implementation, which mirrors the same validation (in particular, the check on line 326): https://github.com/protocolbuffers/protobuf-go/blob/v1.36.6/reflect/protodesc/desc_validate.go#L319

More info in this language spec website: https://protobuf.com/docs/language-spec#maps

So the reflection server has a bug where it generates incorrect descriptors for map types.

[lynxtaa]

parses proto sources, instead of using descriptors compiled by a tool like protoc or buf?

Yeah, I guess it uses https://github.com/protobufjs/protobuf.js under the hood.

Thanks for investigating this issue. Oddly enough both Insomnia and Postman work with this buggy reflection with no issues, perhaps they are less strict.