spark-sql-perf icon indicating copy to clipboard operation
spark-sql-perf copied to clipboard

AnalysisException when calling genData

Open parsifal-47 opened this issue 7 years ago • 0 comments

Hi, on Spark 2.3.2:

scala> tables.genData(
     |     location = rootDir,
     |     format = format,
     |     overwrite = true, // overwrite the data that is already there
     |     partitionTables = true, // create the partitioned fact tables
     |     clusterByPartitionColumns = true, // shuffle to get partitions coalesced into single files.
     |     filterOutNullPartitionValues = false, // true to filter out the partition with NULL key value
     |     tableFilter = "", // "" means generate all tables
     |     numPartitions = 1) 
org.apache.spark.sql.AnalysisException: cannot resolve '`cs_sold_date_sk`' given input columns: [catalog_sales_text.value]; line 8 pos 2;
'RepartitionByExpression ['cs_sold_date_sk], 200
+- Project [value#64]
   +- SubqueryAlias catalog_sales_text
      +- LogicalRDD [value#64], false

parsifal-47 avatar Oct 24 '18 17:10 parsifal-47