Spark-The-Definitive-Guide icon indicating copy to clipboard operation
Spark-The-Definitive-Guide copied to clipboard

Chapter 7 Wrong code in the window function example

Open xuewei4d opened this issue 6 years ago • 2 comments

https://github.com/databricks/Spark-The-Definitive-Guide/blob/38e881406cd424991a624dddb7e68718747b626b/code/Structured_APIs-Chapter_7_Aggregations.scala#L171

First, the column Quantity is parsed as String. It should be IntegerType.

Second orderBy(CustomerId) should be orderBy(desc(Quantity))

xuewei4d avatar May 17 '19 21:05 xuewei4d

want to just open a pull request if you're seeing something wrong?

bllchmbrs avatar May 30 '19 20:05 bllchmbrs

@xuewei4d ===> I ran the below command - dfWithDate.printSchema() resulted in Quantity as an IntegerType root |-- InvoiceNo: string (nullable = true) |-- StockCode: string (nullable = true) |-- Description: string (nullable = true) |-- Quantity: integer (nullable = true) |-- InvoiceDate: string (nullable = true) |-- UnitPrice: double (nullable = true) |-- CustomerID: integer (nullable = true) |-- Country: string (nullable = true) |-- date: date (nullable = true) ===> orderBy("CustomerId") or orderBy($"Quantity".desc) resulted in the same result.

shanmugavel04 avatar Jun 12 '21 21:06 shanmugavel04