sparksql-protobuf icon indicating copy to clipboard operation
sparksql-protobuf copied to clipboard

Published 20 hours ago • verified publisher icon saurfang

Support for Protobuf DynamicMessages

Open jonathanandrade10 opened this issue 5 years ago • 1 comments
trafficstars

Hi, We tested sparksql-protobuf here, works pretty well, but we need to generate the protobuf classes with Protoc .

We have created a feature to pull the latest proto descriptors version from S3 , so that we don't have to keep compiling and including new versions of the compiled ProtoBuf classes. The problem is that the com.google.protobuf.message doesn't contains the method getDescriptor(), it contains getDescriptorForType() instead.

is it possible to create a Support for Protobuf DynamicMessages like this parquet-proto issue ? https://github.com/apache/parquet-mr/pull/414

jonathanandrade10 avatar Jun 16 '20 17:06 jonathanandrade10

maybe? that sounds reasonable. pull request is welcomed. if you are not sure how to get it to work, please open a pull request with a failing test case that demonstrates your use case. it'd be much easier for someone to work from there backward

saurfang avatar Sep 13 '20 01:09 saurfang