Add message_path() to Message trait

[vbkaisetsu]

This branch annotates package names and adds message_path() function to Message trait. This function returns the full path of each proto message.

This is useful for converting each message to Any.

Example:

syntax = "proto3";

package example.proto;

message Example {
    string value = 1;
}

use prost::{DecodeError, Message};
use prost_types::Any;

mod proto {
    include!(concat!(env!("OUT_DIR"), "/example.proto.rs"));
}

fn convert_to_any<T>(message: &T) -> Any where T: Message {
    Any {
        type_url: format!("example.com/{}", T::message_path()),
        value: message.encode_to_vec(),
    }
}

fn convert_from_any<T>(message: &Any) -> Result<T, DecodeError> where T: Message + Default {
    let expected_type_url = format!("example.com/{}", T::message_path());
    if &message.type_url == &expected_type_url {
        T::decode(message.value.as_slice())
    } else {
        Err(DecodeError::new("Invalid type_url"))
    }
}

let message = proto::Example {
    value: "abc".to_string(),
};

let message = convert_to_any(&message);
println!("{:?}", message);
// -> Any { type_url: "example.com/example.proto.Example", value: [10, 3, 97, 98, 99] }

let message: proto::Example = convert_from_any(&message).unwrap();
println!("{:?}", message);
// -> Example { value: "abc" }

[LucioFranco]

I wonder if this can be achieved with https://docs.rs/prost-types/0.8.0/prost_types/struct.FileDescriptorProto.html instead of adding a new fn to the main prost trait which I would like to avoid.

[vbkaisetsu]

message_path() returns only limited information compared to DescriptorProto, so it may need to be modified. However, I think there is no method to make relation between each Message struct and DescriptorProto now. In any case, I think we need to add a new function to get information for the Message trait.

[rdelfin]

@LucioFranco could we allow this change or something like it as a stop gap while a solution using the file descriptor proto is made? I'm tracking thinks like #336 and it seems like finishing a full solution for this will take time. Unfortunately without this, it's impossible to work with Any structs

[LucioFranco]

At work we use any just fine so I don't see why not having this makes it impossible? I am not a huge fan of adding this method so its likely that this PR will not land.

[rdelfin]

@LucioFranco maybe it's more a function of how we use it at my job, but I can't find how, using prost, I can get a type_url for an arbitrary type. I can hard-code it in, but the solution doesn't seem very... Clean. Currently in C++ we just do an Any.Pack() and call it a day. Am I missing some other way of packing these? Frankly, I wouldn't be surprised if I glossed over something important :sweat_smile:

[isaacd9]

At work we use any just fine so I don't see why not having this makes it impossible? I am not a huge fan of adding this method so its likely that this PR will not land.

How do you serialize messages to and from google.protobuf.Any? It looks to me like the prost::Message type doesn't provide a way to retrieve the type_url or Descriptor for a given message to specify the type of the encoded Any message. Is there another way to pack them effectively into the Any type?

[oblique]

@LucioFranco Can you show an example on how do you use it?

[isaacd9]

At work we use any just fine so I don't see why not having this makes it impossible? I am not a huge fan of adding this method so its likely that this PR will not land.

How do you serialize messages to and from google.protobuf.Any? It looks to me like the prost::Message type doesn't provide a way to retrieve the type_url or Descriptor for a given message to specify the type of the encoded Any message. Is there another way to pack them effectively into the Any type?

The (awkward) solution I've been using to make this work is define a new trait with a type_url method:

pub trait ProstMessageExt {
    fn type_url() -> String;
}

and then implement it on the messages which I care to pack into a google.protobuf.Any.

let encoded = prost_types::Any {
    type_url: my_message::type_url(),
    value: element.encode_to_vec(),
};

This works fine but requires a definition on every message type, making it a bit hard to write a library, and expanding trait bounds to prost::Message + ProstMessageExt.

[oblique]

@isaacd9 This solution is indeed awkward and has the following problems:

• You can always have a typo in the url string.
• If a message is renamed or moved under another package, you will not get any compilation errors if you forgot to update the url string.

In my opinion this is very error prune.

[oblique]

@vbkaisetsu Correct me if I'm wrong, but shouldn't type.googleapis.com/ always be the prefix in type url?

[vbkaisetsu]

@oblique The prefix is required. I think it should not be given automatically, but should be specified by the user.

In Java and Python, this prefix is an optional argument. When it is not specified, type.googleapis.com/ will be set. https://developers.google.com/protocol-buffers/docs/reference/java/com/google/protobuf/Any https://googleapis.dev/python/protobuf/latest/google/protobuf/any_pb2.html

[vbkaisetsu]

• If a message is renamed or moved under another package, you will not get any compilation errors if you forgot to update the url string.

Yes, it is. The path part of type_url should be generated automatically by prost, and specifying it manually is just a cause of bugs.

[wangfenjin]

I really suggest the team to consider this seriously, this kind of APIs are widely used and very useful in daily development. Previously it's discussed in #255 and #336 , both the PRs are rejected. People need to hack around this and even create another crate https://github.com/fdeantoni/prost-wkt for this.

[LucioFranco]

Yeah, I think we can try to merge something to support this in the next release. I would like to formalize a proper road map. I'd like to evaluate our options here and educate myself on how other languages support this behavior.

[caspermeijn]

Is this still needed? The trait Name exists now and is had the type_url method.

Does that cover the use cases for this PR? https://docs.rs/prost/latest/prost/trait.Name.html#method.type_url

[oblique]

Not needed anymore. The usecase was to convert Any to the message type. With Name and Any::to_msg this can be done now.

[caspermeijn]

I close this issue as it seems to be solved. Please reopen if that is incorrect.