succinct
succinct copied to clipboard
Binary Type not Supported in a Succinct Data Frame
val df = spark.sql("SELECT * from table_name limit 100") df.write.format("edu.berkeley.cs.succinct.sql").save("/Users/hello/df")
This is what happens:
java.lang.IllegalArgumentException: Unexpected type. BinaryType at edu.berkeley.cs.succinct.sql.SuccinctTableRDD$$anonfun$7.apply(SuccinctTableRDD.scala:223) at edu.berkeley.cs.succinct.sql.SuccinctTableRDD$$anonfun$7.apply(SuccinctTableRDD.scala:213) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234) at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33) at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186) at scala.collection.TraversableLike$class.map(TraversableLike.scala:234) at scala.collection.mutable.ArrayOps$ofRef.map(ArrayOps.scala:186) at edu.berkeley.cs.succinct.sql.SuccinctTableRDD$.min(SuccinctTableRDD.scala:213) at edu.berkeley.cs.succinct.sql.SuccinctTableRDD$.apply(SuccinctTableRDD.scala:157) at edu.berkeley.cs.succinct.sql.SuccinctTableRDD$.apply(SuccinctTableRDD.scala:175) at edu.berkeley.cs.succinct.sql.package$SuccinctDataFrame.saveAsSuccinctTable(package.scala:29) at edu.berkeley.cs.succinct.sql.DefaultSource.createRelation(DefaultSource.scala:46) at org.apache.spark.sql.execution.datasources.DataSource.write(DataSource.scala:429)
@anuragkh
@anuragkh Any update on this?