spark-tfrecord icon indicating copy to clipboard operation
spark-tfrecord copied to clipboard

Support for ShortType, ByteType

Open echan5 opened this issue 5 years ago • 4 comments

Hi @junshi15 ,

I was wondering if I could put in a request to support ShortType and ByteType? I'm getting errors like Cannot convert field to unsupported data type ShortType which I believe is coming from https://github.com/linkedin/spark-tfrecord/blob/master/src/main/scala/com/linkedin/spark/datasources/tfrecord/TFRecordSerializer.scala

Thank you!

echan5 avatar Sep 04 '20 01:09 echan5

Yes, of course, please send your PR.

junshi15 avatar Sep 04 '20 01:09 junshi15

Unfortunately, I'm not familiar with Scala. I could probably guess some of it but I'm not sure if you'd prefer someone who is familiar with Scala coding that?

echan5 avatar Sep 04 '20 01:09 echan5

No problem. Scala is not very different from Java.

junshi15 avatar Sep 04 '20 01:09 junshi15

I code in python mostly, so thank you, much appreciated!

echan5 avatar Sep 04 '20 01:09 echan5