elephant-bird icon indicating copy to clipboard operation
elephant-bird copied to clipboard

stackoverflow error occured while using ProtobufPigLoader on nested proto.

Open CCv5 opened this issue 5 years ago • 0 comments

I am reading protobuf data using ProtobufPigLoader, it will fail when my proto have nested defination like following example, the error log is attched.

I didn't set any value on this field, It is the default value. it would be OK after removing this specific field. checking the code (ProtobufToPig.java:192 and ProtobufToPig.java:212), it is definatly an endless recursive call.

Any Suggestion?

message A { repeated A sub_A = 1; ... }

1 Pig Stack Trace 2 --------------- 3 ERROR 2998: Unhandled internal error. null 4 5 java.lang.StackOverflowError 6 at java.util.HashMap.hash(HashMap.java:338) 7 at java.util.HashMap.put(HashMap.java:611) 8 at org.apache.pig.impl.plan.NodeIdGenerator.getNextNodeId(NodeIdGenerator.java:48) 9 at org.apache.pig.impl.logicalLayer.CanonicalNamer.getNewName(CanonicalNamer.java:33) 10 at org.apache.pig.impl.logicalLayer.schema.Schema$FieldSchema.(Schema.java:155) 11 at com.twitter.elephantbird.pig.util.ProtobufToPig.singleFieldToFieldSchema(ProtobufToPig.java:241) 12 at com.twitter.elephantbird.pig.util.ProtobufToPig.toSchema(ProtobufToPig.java:192) 13 at com.twitter.elephantbird.pig.util.ProtobufToPig.messageToFieldSchema(ProtobufToPig.java:212) 14 at com.twitter.elephantbird.pig.util.ProtobufToPig.toSchema(ProtobufToPig.java:190) 15 at com.twitter.elephantbird.pig.util.ProtobufToPig.messageToFieldSchema(ProtobufToPig.java:212) 16 at com.twitter.elephantbird.pig.util.ProtobufToPig.toSchema(ProtobufToPig.java:190) 17 at com.twitter.elephantbird.pig.util.ProtobufToPig.messageToFieldSchema(ProtobufToPig.java:212) 18 at com.twitter.elephantbird.pig.util.ProtobufToPig.toSchema(ProtobufToPig.java:190) 19 at com.twitter.elephantbird.pig.util.ProtobufToPig.messageToFieldSchema(ProtobufToPig.java:212) 20 at com.twitter.elephantbird.pig.util.ProtobufToPig.toSchema(ProtobufToPig.java:190) 21 at com.twitter.elephantbird.pig.util.ProtobufToPig.messageToFieldSchema(ProtobufToPig.java:212) 22 at com.twitter.elephantbird.pig.util.ProtobufToPig.toSchema(ProtobufToPig.java:190) 23 at com.twitter.elephantbird.pig.util.ProtobufToPig.messageToFieldSchema(ProtobufToPig.java:212) 24 at com.twitter.elephantbird.pig.util.ProtobufToPig.toSchema(ProtobufToPig.java:190)

CCv5 avatar Oct 08 '18 11:10 CCv5