Clarification on loading data folder having multiple CSV file from local hard drive

[vishnu-muraly]

Hi, I'm really stuck with this section of Spark book. staticDataFrame = spark.read.format("csv")
.option("header", "true")
.option("inferSchema", "true")
.load("/mnt/defg/retail-data/by-day/*.csv")

1. I'm not able to understand the "load("/mnt/...") section. I have downloaded the data to my local drive. But now the issue is on loading the data. How to load the data ?

2. Is the mnt/defg being done via S3 ? or by any other method !

[hiteshtiwari]

Hi,

If file is in your local drive. Then try file://<local_file path>

Hitesh Tiwari

On Fri, 19 Apr 2019, 15:31 Vishnu Murali, [email protected] wrote:

Hi, I'm really stuck with this section of Spark book. staticDataFrame = spark.read.format("csv") .option("header", "true") .option("inferSchema", "true") .load("/mnt/defg/retail-data/by-day/.csv")*

I'm not able to understand the "load("/mnt/...") section. I have downloaded the data to my local drive. But now the issue is on loading the data. How to load the data ? 2.

Is the mnt/defg being done via S3 ? or by any other method !

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/databricks/Spark-The-Definitive-Guide/issues/38, or mute the thread https://github.com/notifications/unsubscribe-auth/ACV2AHLGGMTSAPEBJFQSWPDPRGJ6RANCNFSM4HHDHFIA .