serializing protobuffer messages using mergefrom fails when enums_by_name enabled

[vrecan]

If you take a normal protobuffer message and serialize it to bytes:

Search query = Search.newBuilder().setQueryFilter(queryFilter).build();
byte[] protoBytes = query.toByteArray();

and then try to use mergefrom :

      Search.Builder build = Search.newBuilder();
      ProtobufIOUtil.mergeFrom(proto, build, SchemaQuery.Search.MERGE);

It will fail to parse enum values with an error:

java.lang.IllegalArgumentException: No enum constant com.logrhythm.justonemorething.query.Query.FilterItemTypes.

if you remove the enums_by_name option everything works as expected except when I serialize them into json they have the enum int values which I don't want. I think I can work around this by using the runtime version but this seems like a pretty big compatibility bug.

This means with this option enabled it is no longer compatible with standard protobuffer messages and everything has to use protostuff. This kind of defeats the purpose of using protobuffers.

[dyu]

Protobuf writes enums by their number/ordinal. That's what protostuff does. Json is a supported format which recieves that number and writes a json number.

[dyu]

enums_by_name is an api-level change which affects all the supported formats. You want only enums_by_name enabled for json but that is not possible.

[vrecan]

Well it does work if I use just the protostuff serialize and deserialize from protobuf to json. And it seems like the major point of java_v2protoc_schema is to have compatibility with standard protobuffers. This breaks that, which seems like a pretty big bug.

[vrecan]

Why can't we just add a feature from the schema that's generated to be able to translate the numeric types that you get over the wire to the named types when this option is enabled, keeping compatibility with standard protobuffers?

[vrecan]

When I do this:

      Search.Builder build = Search.newBuilder();
      ProtobufIOUtil.mergeFrom(proto, build, SchemaQuery.Search.MERGE);
      Search search = build.build();
      ByteArrayOutputStream out = new ByteArrayOutputStream();
      JsonIOUtil.writeTo(out, search, SchemaQuery.Search.WRITE, false);

it works correctly if I give it a byte[] that's been generated from protostuff, the json output has names in place of enums and everything is great. The only problem is when a standard protobuffer byte[] comes in, it can't be translated because it's expecting the enums to be a string. Why can't we just detect this scenario that the data coming in is an int and not a string. And then use the standard logic to parse it into a message?

[vrecan]

I got this to work for me by just using the standard protoc serialization and then taking that and using the schema.

      ByteArrayOutputStream out = new ByteArrayOutputStream();
      Search search = Search.newBuilder().mergeFrom(proto).build();
      JsonIOUtil.writeTo(out, search, SchemaQuery.Search.WRITE, false);

If anyone else is running into this issue.

[kshchepanovskyi]

Hm, I did not know about this enums_by_name side-effect. It should be possible to reimplement "enums by name" in different way, similar to io.protostuff.JsonOutput#numeric.

[kshchepanovskyi]

Fix for this issue requires too much changes. It will not be a part of 1.3.0 release.

[kshchepanovskyi]

We can not fix this without breaking backward compatibility (need to add new methods to Schema interface). Moved to milestone "2.0".