prost

prost copied to clipboard

I'm struggling to figure out how to access EnumValueOption extensions on an enum. For example, given

proto/demo.proto:

syntax = "proto3";

import "google/protobuf/descriptor.proto";

package demo;

extend google.protobuf.EnumValueOptions {
  optional uint32 len = 50000;
}

enum Foo {
  None = 0 [(len) = 0];
  One = 1 [(len) = 1];
  Two = 2 [(len) = 2];
}

and

build.rs:

use std::path::PathBuf;

fn main() {
    let out_dir =
        PathBuf::from(std::env::var("OUT_DIR").expect("OUT_DIR environment variable not set."));

    prost_build::Config::new()
        .file_descriptor_set_path(out_dir.join("file_descriptor_set.bin"))
        .compile_protos(&["proto/demo.proto"], &["proto"])
        .unwrap_or_else(|e| panic!("Failed to compile protos {:?}", e));
}

I suspect I can somehow access the len options using the FileDescriptorSet, but I can't figure out how.

src/lib.rs:

use prost::Message;
use prost_types::FileDescriptorSet;

include!(concat!(env!("OUT_DIR"), concat!("/demo.rs")));

pub fn parse_file_descriptor_set() -> FileDescriptorSet {
    let file_descriptor_set_bytes =
        include_bytes!(concat!(env!("OUT_DIR"), "/file_descriptor_set.bin"));
    prost_types::FileDescriptorSet::decode(&file_descriptor_set_bytes[..]).unwrap()
}

#[cfg(test)]
mod tests {
    use crate::parse_file_descriptor_set;

    #[test]
    fn get_len() {
        let set = parse_file_descriptor_set();
        for f in &set.file {
            dbg!(&f.extension);
        }
        todo!()
    }
}

@shradej1 were you able to figure this out?

Nope, still hoping for a response.

I think the right place to look also would be in protoc what it generates since FileDescriptorSet etc is all just generated from protoc I don't know off the top of my head what gets generated into there beyond what we use.

I wonder if this is just missing functionality. If I change get_len() as follows

fn get_len(foo: Foo) -> u32 {
        let set = parse_file_descriptor_set();
        for f in &set.file {
            for ext in &f.extension {
                dbg!(ext);
            }
            for e in &f.enum_type {
                for v in &e.value {
                    dbg!(&v.options);
                }
            }
        }
        todo!()
    }

I see

[src/lib.rs:22] ext = FieldDescriptorProto {
    name: Some(
        "len",
    ),
    number: Some(
        50000,
    ),
    label: Some(
        Optional,
    ),
    r#type: Some(
        Uint32,
    ),
    type_name: None,
    extendee: Some(
        ".google.protobuf.EnumValueOptions",
    ),
    default_value: None,
    oneof_index: None,
    json_name: Some(
        "len",
    ),
    options: None,
    proto3_optional: Some(
        true,
    ),
}

so the extension is being parsed. I don't see the actual len values being stored anywhere though. However, if I add a deprecated option to one of the enum values - None = 0 [(len) = 10, deprecated = true];, it does show up.

[src/lib.rs:29] &v.options = Some(
    EnumValueOptions {
        deprecated: Some(
            true,
        ),
        uninterpreted_option: [],
    },
)

I feel like len should be captured in that uninterpreted_option, which I should be able to parse via (missing?) functionality in the FieldDescriptorProto extension? I might be able to take a peek and figure out where deprecated is being set, and what would actually end up in the uninterpreted_option of EnumValueOption.

I may be missing something. I think that extension value is actually encoded in the serialized message, in which case I'd need the byte stream - not just the enum value - to retrieve it.

I feel like len should be captured in that uninterpreted_option, which I should be able to parse via (missing?) functionality in the FieldDescriptorProto extension? I might be able to take a peek and figure out where deprecated is being set, and what would actually end up in the uninterpreted_option of EnumValueOption.

It looks extension options are actually serialized as unknown fields, which prost doesn't currently capture (#574 might change that)

For example, the text format of your example file descriptor is:

file {
  name: "foo.proto"
  package: "demo"
  dependency: "google/protobuf/descriptor.proto"
  enum_type {
    name: "Foo"
    value {
      name: "None"
      number: 0
      options {
        50000: 0
      }
    }
    value {
      name: "One"
      number: 1
      options {
        50000: 1
      }
    }
    value {
      name: "Two"
      number: 2
      options {
        50000: 2
      }
    }
  }
  extension {
    name: "len"
    extendee: ".google.protobuf.EnumValueOptions"
    number: 50000
    label: LABEL_OPTIONAL
    type: TYPE_UINT32
    json_name: "len"
    proto3_optional: true
  }
  syntax: "proto3"
}

Which does technically contain enough information to reconstruct the option, but you have to inspect the extension fields as well.

My crate prost-reflect is able to retrieve the option values, for example:

let file_descriptor_set_bytes = include_bytes!(concat!(env!("OUT_DIR"), "/file_descriptor_set.bin"));

let descriptors = prost_reflect::DescriptorPool::decode(file_descriptor_set_bytes.as_ref()).unwrap();
let file_descriptor_set = descriptors
    .get_message_by_name("google.protobuf.FileDescriptorSet")
    .unwrap();

let mut dynamic_message = prost_reflect::DynamicMessage::new(file_descriptor_set);
dynamic_message.merge(bytes.as_slice()).unwrap();

let extension = descriptors
    .get_message_by_name("google.protobuf.EnumValueOptions")
    .unwrap()
    .extensions()
    .find(|ext| ext.name() == "len")
    .unwrap();

assert_eq!(dynamic_message
    .get_field_by_name("file")
    .unwrap()
    .as_list()
    .unwrap()[1]
    .as_message()
    .unwrap()
    .get_field_by_name("enum_type")
    .unwrap()
    .as_list()
    .unwrap()[0]
    .as_message()
    .unwrap()
    .get_field_by_name("value")
    .unwrap()
    .as_list()
    .unwrap()[1]
    .as_message()
    .unwrap()
    .get_field_by_name("options")
    .unwrap()
    .as_message()
    .unwrap()
    .get_extension(&extension)
    .as_ref(), &Value::U32(1));

Ideally there would be solution to bind into a strongly-typed message to avoid the nasty dynamic message inspection, but I'm not aware of any crates that provide that