javacpp-presets

javacpp-presets copied to clipboard

I'm constructing ONNX models via javacpp-presets, but I'm getting errors when my message is past a certain size.

In the below code, parseFrom is imported from com.google.protobuf.Any, and Builder is my class that calls the javacpp-presets ONNX protobuf APIs. For completeness, it's in a gist here.

This works:

    val builder = Builder("Relu123456789012345678901234")
    builder.addConst("c", constData, inputShape)
    builder.addNode(Vector("c"), Vector("out"), "Relu")
    builder.addOutput("out")
    val model = builder.build()  // Returns a ModelProto
    parseFrom(model.SerializeAsString.getString.getBytes)

but this fails on the last line (the difference is that the string in the first line is one character longer):

    val builder = Builder("Relu1234567890123456789012345")
    builder.addConst("c", constData, inputShape)
    builder.addNode(Vector("c"), Vector("out"), "Relu")
    builder.addOutput("out")
    val model = builder.build()  // Returns a ModelProto
    parseFrom(model.SerializeAsString.getString.getBytes)

The specific error is:

[info]   com.google.protobuf.InvalidProtocolBufferException: While parsing a protocol message, the input ended unexpectedly in the middle of a field.  This could mean either that the input has been truncated or that an embedded message misreported its own length.
[info]   at com.google.protobuf.InvalidProtocolBufferException.truncatedMessage(InvalidProtocolBufferException.java:86)
[info]   at com.google.protobuf.CodedInputStream$ArrayDecoder.readRawBytes(CodedInputStream.java:1281)
[info]   at com.google.protobuf.CodedInputStream$ArrayDecoder.readBytes(CodedInputStream.java:935)
[info]   at com.google.protobuf.UnknownFieldSet$Builder.mergeFieldFrom(UnknownFieldSet.java:537)
[info]   at com.google.protobuf.GeneratedMessageV3.parseUnknownFieldProto3(GeneratedMessageV3.java:305)
[info]   at com.google.protobuf.Any.<init>(Any.java:114)
[info]   at com.google.protobuf.Any.<init>(Any.java:75)
[info]   at com.google.protobuf.Any$1.parsePartialFrom(Any.java:920)
[info]   at com.google.protobuf.Any$1.parsePartialFrom(Any.java:915)
[info]   at com.google.protobuf.AbstractParser.parsePartialFrom(AbstractParser.java:163)
[info]   ...

The javacpp-presets docs show that ModelProto can take a size parameter; I have tried setting this to a large number but it doesn't solve the problem.

How can I construct a valid protobuf message using the javacpp-presets ONNX APIs?

cc @EmergentOrder

I was able to reproduce this issue, with the latest ONNX preset and a few versions of protobuf . Also confirmed I cannot reproduce the same from Python. I found someone else reporting this same issue here: https://groups.google.com/forum/#!topic/protobuf/hOJV6PrLwy0 and several others complaining of the same error, but not sure of the trigger. So, it looks like a bug in https://github.com/protocolbuffers/protobuf/tree/master/java