java icon indicating copy to clipboard operation
java copied to clipboard

Could we supply the method that convert spark sql dataframe to tensorflow-java ByteNdArray object ?

Open mullerhai opened this issue 3 years ago • 2 comments
trafficstars

Hi : Now I etl the raw data use spark sql generate the dataframe , I want to convert the spark dataframe object to ByteNdArray object for tensorflow model input

mullerhai avatar May 25 '22 09:05 mullerhai

Converting from a data frame to a byte array has a lot of options on how you represent the data, columns and encoding. Do you have something specific in mind for each of those?

Craigacp avatar May 25 '22 13:05 Craigacp

Converting from a data frame to a byte array has a lot of options on how you represent the data, columns and encoding. Do you have something specific in mind for each of those? -- dataframe -- text file ---ByteNdArray is easy ,but it need to write file to disk , but I want to use the tools that only need read memory dataframe cache meta-convert to ByteNdArray

mullerhai avatar May 25 '22 14:05 mullerhai