ch 6 map -> create_map

[dpuddu86]

https://github.com/databricks/Spark-The-Definitive-Guide/blob/38e881406cd424991a624dddb7e68718747b626b/code/Structured_APIs-Chapter_6_Working_with_Different_Types_of_Data.py#L313

It seems SCALA version, same as #L319

[bllchmbrs]

Sorry, what's the issue? Could you open a pull request with the change?

[dpuddu86]

Hi, lines 313 to 320 seem not working using pySpark. The map command seems SCALA and not pyspark language, I think it should be replaced with create_map. Even the book "Spark, The Definitive Guide" seems affected by the same error at pag 109.

I'm sorry but I'm not able to open a pull request. I'm just using the code as a reference for a tutorial on Databricks community available online. This is the first issue I report.

The following lines may help to understand. L307(WORKING using create_map) df.select(create_map(col("Description"), col("InvoiceNo")).alias("complex_map"))
.show(2)

L313 (NOT WORKING, here we have map instead of create_map of line 307) df.select(map(col("Description"), col("InvoiceNo")).alias("complex_map"))
.selectExpr("complex_map['WHITE METAL LANTERN']").show(2)

L319 (NOT WORKING, here we have map instead of create_map of line 307) df.select(map(col("Description"), col("InvoiceNo")).alias("complex_map"))
.selectExpr("explode(complex_map)").show(2)

[JoeKastner]

Just came across this as well - I'll submit a PR with the fix.