Calvin Neo
Calvin Neo
- 视频推荐： 标题：Functional Programming Fundamentals 链接：https://channel9.msdn.com/Series/C9-Lectures-Erik-Meijer-Functional-Programming-Fundamentals/Lecture-Series-Erik-Meijer-Functional-Programming-Fundamentals-Chapter-1 作者：Erik Meijer 标签（选填）：原书也很棒，这里推荐看视频，会更清楚
@jilugulu I did not compile Spark2.4.4, I download it from https://spark.apache.org/downloads.html. @ShubhamGupta29 How can I locate Dependency.scala?
@ShubhamGupta29 This is my Dependency.scala. [Dependencies.scala.zip](https://github.com/linkedin/dr-elephant/files/3978542/Dependencies.scala.zip) I modified some conf files including change spark-default.conf ``` spark.master spark://localhost:7077 spark.eventLog.enabled false spark.eventLog.dir hdfs://localhost:9000/user/spark/appHist spark.history.fs.logDirectory /.../spark-2.4.4-bin-hadoop2.7/conf/history/spark-events ``` I also `export SPARK_MASTER_HOST=localhost`, otherwise it...
@ShubhamGupta29 Actually, I run `echo $SCALA_HOME` and it shows my scala is 2.13.1
@ShubhamGupta29 2.10.4, and I didn't modify this file
I read some [introduction](https://engineering.linkedin.com/blog/2016/04/dr-elephant-open-source-self-serve-performance-tuning-hadoop-spark) of drelephant, to my opinion, it requires metric from Spark/Hadoop. As far as I know, spark's eventlog is a JSON file, so I can just download...
Yes, I have not implemented module yet...
/cc @BusyJay @breezewish @tonyxuqqi @5kbpers