ranking icon indicating copy to clipboard operation
ranking copied to clipboard

How to store spark dataframe as ELWC tfrecord?

Open ZhouM1118 opened this issue 2 years ago • 0 comments

My team uses spark for data preprocessing, and now I need to store the dataframe in spark in HDFS in tfrecord format of ELWC, we use writeDF.write.mode(writeMode).format("tfrecord").option("recordType", "Example") to save dataframe as tfrecord format, but it only support Example or SequenceExample tfrecord format, so how to store spark dataframe as ELWC tfrecord? Thank you for your help!

ZhouM1118 avatar Jan 12 '22 09:01 ZhouM1118