Dr Elephant currently supports spark 2.3？

[shangwen]

dr-elephant currently supports spark 2.3? I have a compilation exception

[info] Resolving org.jacoco#org.jacoco.agent;0.7.1.201405082137 ...
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  ::          UNRESOLVED DEPENDENCIES         ::
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  :: org.apache.spark#spark-core_2.10;2.3.0: not found
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn] 
[warn]  Note: Unresolved dependencies path:
[warn]          org.apache.spark:spark-core_2.10:2.3.0 (/data/wallytang/workspace/dr-elephant/build.sbt#L28-29)
[warn]            +- com.linkedin.drelephant:dr-elephant_2.10:2.1.7

[bteeuwen]

Yes I also have this question. We are using Spark 2.3.1 and would like to try out Dr. Elephant after hearing the nice talk at Strata this year. Any news about the compatibility? I heard that LinkedIn is using a patched Spark version internally, so maybe there is no direct interest to ensure compatibility for the latest open source version. If this is not prioritized, it would be good to know what steps are missing so maybe someone can step in from the community?

[20100507]

Yes I also have this question.I hope reslove this or tell us! thanks!

[wtx626]

I am sorry to tell you that I have four compilation errors when I use Spark 2.2.0 and Hadoop 2.6.0.

[varunsaxena]

We internally use custom Spark 2.x versions in LinkedIn. The relevant code is in customSHSWork branch but as the name indicates this is based on custom internal APIs' and would not work for open source Spark versions. You will have to write a new fetcher for your use case. It would be great if you can contribute it.