Michael Bernadskiy

Hi Rahul, I am getting an exception with your fix. I have compiled elephant-bird for hadoop2 and bumped protobuf to 2.5.0. I will try to get more debug info next...

Rahul, thanks a lot. I'll test it first thing tomorrow morning.

Rahul, great - the previous exception is gone, but I am seeing another one: Caused by: java.lang.ClassCastException: Logentries$Impression$NestedMessageType cannot be cast to com.google.protobuf.Message$Builder at com.twitter.elephantbird.hive.serde.ProtobufStructObjectInspector.getStructFieldData(ProtobufStructObjectInspector.java:193) at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorConverters$StructConverter.convert(ObjectInspectorConverters.java:394) at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorConverters$ListConverter.convert(ObjectInspectorConverters.java:337) at...

Hi Rahul, I have created a patch that does not cause exceptions: https://gist.github.com/mikebern/825812f0de95a2f03dd5 I will need to verify that the data is correct. I'll try to add unit tests after...

This is not a bug in your code. You code enables a use case that didn't work before at all with the stock Hive. Specifically, it allows to run queries...

Thanks, Rahul. I will try tomorrow morning.

I modified the code slightly: public boolean equals(Object o) { if (!(o instanceof ProtobufStructObjectInspector)) { return false; } ProtobufStructObjectInspector that = (ProtobufStructObjectInspector)o; boolean res = descriptor.equals(that.descriptor); boolean res2 = builder.equals(that.builder);...

Sorry, was busy with getting Spark/Shark working with EB. I will send an update tomorrow.

I check the descriptor - in all calls to the equal method the compared objects are actually the same object (and different for the Builder objects). Looks like Descriptor class...