Introspection support

[peng1999]

Is it possible to define a fn descriptor(&self) -> MessageDescriptor function in trait Message in order to get Message name? Reference: https://developers.google.com/protocol-buffers/docs/reference/cpp/google.protobuf.descriptor#Descriptor

[flxo]

I have the need for getting a string representation of a Message containing package and message name. Looking at prost_derive I think I understood that it's not possible to obtain the module in a proc macro since only the struct definition is passed.

The full set of MessageDescriptor is IMHO only possible with generating some more code. Easy but then there would be a gab between generated models and normal structs that just derive Message.

[aajtodd]

The readme states

Does not include support for runtime reflection or message descriptors.

I'm not sure if the reason is simply because it wasn't important to start with or if there is a technical hurdle (or it's just a non-goal which is fine too). I'd like to see the reflection API supported fully though personally.

Perhaps someone could elaborate if they know more about this.

[danburkert]

I'm in support of adding a reflection API, but it needs a significant amount of design. It would be nice to have a descriptor method which yields a Descriptor instance, but as @flxo pointed out, that seems tricky to do without inlining said descriptor into the generated code.

At a bit higher level, I'd love to get some feedback around usecases - I've personally never needed to use reflection with Protobuf, and I don't know what the full spectrum of considerations are. I understand that writing a message proxy which introspects the messages is one usecase, but I'm sure there are more.

Finally, I think a coherent design is necessary to avoid blindly copying what the C++ API does - a primary design goal of prost has been to do protobufs in a way that's idiomatic to Rust, and I'd be concerned that copying the C++ introspection APIs would result in an unintuitive (and large!) interface.

[aajtodd]

Thanks @danburkert for clarifying your stance, I think it helps others know it would be a welcome addition assuming the design can be addressed appropriately.

As far as other use cases, an implementation of the Common Expression Language would require a reflection API.

[aajtodd]

@flxo I think I see what you are saying re: TokenStream not having package info. Is this really a problem though? Couldn't it just be an additional argument or trait to derive where the user manually specifies the package info? For structs that derive Message there is no way to know the package info anyway since the definition wasn't generated from a .proto file so I would think it would have to be supplied by the user.

That makes the most sense to me. It will be more work for the user that is just deriving Message whereas the generated versions would automatically implement the descriptor interfaces.

Thoughts? I'm not an expert on introspection support but I'm going to have to dive into it a bit. I'm willing to maybe invest some time exploring this but some input would be helpful.

[gklijs]

I'd like to mention an use case. Confluent recently added protobuf support to the schema registry. As a consumer of a topic, this means you get both the bytes of the massage, and an ID, with with you can get the protobuf definition used to create the message. To support this in rust, currently it would be a two step plan, where you first check the types used, generate the rust files, then compile and start consuming. It would be nice if there was a way to dynamicly handle the data.

[mattoni]

I've got a use case as well. I'm doing custom typed error messages that are defined in the proto. When I receive an error status, I can access the appropriate data from the metadata binary and reconstruct the type using prost::message::Message::decode(). The issue is, the key for these is defined as the descriptor for the type, which I have no way of getting and have resorted to hard coding which is not ideal.

Another member of my team is using the Java library, and they have a function that is doing something like this:

  public static final String BINARY_HEADER_SUFFIX = "-bin";  

  public static <T extends Message> Metadata.Key<T> keyForProto(T instance) {
    return Metadata.Key.of(
        instance.getDescriptorForType().getFullName() + Metadata.BINARY_HEADER_SUFFIX,
        metadataMarshaller(instance));
  }

which they're using to achieve the same goal. If we could support getting the descriptor for a struct that'd be awesome.

[WENPIN1]

How about 'prost-reflect' craft ? https://github.com/andrewhickman/prost-reflect https://docs.rs/prost-reflect/0.10.2/prost_reflect/trait.ReflectMessage.html