dr-elephant
dr-elephant copied to clipboard
Published
20 hours ago •
linkedin
linkedin
Failed to compile Dr Elephant
Hi,
I am trying to set up Dr Elephant on my local system. But it is failing with the following errors:-
1) java.lang.IllegalStateException: impossible to get artifacts when data has not been loaded. IvyNode = org.slf4j#slf4j-api;1.6.6
2) cd: target/universal: No such file or directory
Full stack trace
➜ dr-elephant git:(master) ✗ ./compile.sh ./compile.conf
Checking for required programs...
[SUCCESS] Program requirement is fulfilled!
Using config file: ./compile.conf
Reading from config file...
Hadoop Version : 3.1.0
Spark Version : 2.3.1
Other opts set : -Dsbt.repository.config=app-conf/resolver.conf
############################################################################
npm installation found, we'll compile with the new user interface
############################################################################
+ sleep 3
+ ember_assets=/Users/R2D2/Desktop/dr-elephant/public/assets
+ ember_resources_dir=/Users/R2D2/Desktop/dr-elephant/public/assets/ember
+ ember_web_directory=/Users/R2D2/Desktop/dr-elephant/web
+ cd /Users/R2D2/Desktop/dr-elephant/web
+ npm install
audited 30093 packages in 11.888s
found 168 vulnerabilities (72 low, 39 moderate, 57 high)
run `npm audit fix` to fix them, or `npm audit` for details
+ node_modules/bower/bin/bower install
+ node_modules/ember-cli/bin/ember build --prod
WARNING: Node v10.9.0 has currently not been tested against Ember CLI and may result in unexpected behaviour.
Could not start watchman; falling back to NodeWatcher for file system events.
Visit http://ember-cli.com/user-guide/#watchman for more info.
Built project successfully. Stored in "dist/".
File sizes:
- dr-elephant.css: 8.61 KB (2.49 KB gzipped)
- dr-elephant.js: 277.75 KB (28.34 KB gzipped)
- aggregated-metrics.css: 207 B (156 B gzipped)
- application-menu.css: 334 B (218 B gzipped)
- application.css: 34 B (54 B gzipped)
- breadcrumbs.css: 1.36 KB (566 B gzipped)
- dashboard-header.css: 609 B (340 B gzipped)
- heuristic-detail.css: 127 B (109 B gzipped)
- heuristics-summary.css: 536 B (281 B gzipped)
- jobs-severity.css: 333 B (202 B gzipped)
- loading-panel.css: 129 B (126 B gzipped)
- navbar.css: 920 B (282 B gzipped)
- not-found.css: 76 B (74 B gzipped)
- search-panel.css: 39 B (59 B gzipped)
- search.css: 569 B (300 B gzipped)
- user-detail.css: 988 B (435 B gzipped)
- user-tabs.css: 725 B (321 B gzipped)
- user-tags.css: 76 B (75 B gzipped)
- vendor.css: 143.17 KB (24.06 KB gzipped)
- vendor.js: 1.21 MB (355.98 KB gzipped)
+ rm -r /Users/R2D2/Desktop/dr-elephant/public/assets/ember
+ mkdir /Users/R2D2/Desktop/dr-elephant/public/assets/ember
+ cp dist/assets/dr-elephant.css /Users/R2D2/Desktop/dr-elephant/public/assets/ember/
+ cp dist/assets/dr-elephant.js /Users/R2D2/Desktop/dr-elephant/public/assets/ember/
+ cp dist/assets/vendor.js /Users/R2D2/Desktop/dr-elephant/public/assets/ember/
+ cp dist/assets/vendor.css /Users/R2D2/Desktop/dr-elephant/public/assets/ember/
+ cp -r dist/fonts /Users/R2D2/Desktop/dr-elephant/public/assets/
+ cd /Users/R2D2/Desktop/dr-elephant
+ trap exit SIGINT SIGTERM
+ start_script=/Users/R2D2/Desktop/dr-elephant/scripts/start.sh
+ stop_script=/Users/R2D2/Desktop/dr-elephant/scripts/stop.sh
+ app_conf=/Users/R2D2/Desktop/dr-elephant/app-conf
+ pso_dir=/Users/R2D2/Desktop/dr-elephant/scripts/pso
+ rm -rf /Users/R2D2/Desktop/dr-elephant/dist
+ mkdir dist
+ play_command -Dhadoopversion=3.1.0 -Dsparkversion=2.3.1 -Dsbt.repository.config=app-conf/resolver.conf clean test compile dist
+ type activator
activator is /Users/R2D2/Downloads/activator-dist-1.3.12/bin/activator
+ activator -Dhadoopversion=3.1.0 -Dsparkversion=2.3.1 -Dsbt.repository.config=app-conf/resolver.conf clean test compile dist
[info] Loading project definition from /Users/R2D2/Desktop/dr-elephant/project
[info] Set current project to dr-elephant (in build file:/Users/R2D2/Desktop/dr-elephant/)
[success] Total time: 0 s, completed Sep 10, 2018 12:03:06 PM
[info] Updating {file:/Users/R2D2/Desktop/dr-elephant/}dr-elephant...
[info] Resolving org.apache.spark#spark-core_2.10;2.3.1 ...
[warn] module not found: org.apache.spark#spark-core_2.10;2.3.1
[warn] ==== local: tried
[warn] /Users/R2D2/.ivy2/local/org.apache.spark/spark-core_2.10/2.3.1/ivys/ivy.xml
[warn] ==== activator-launcher-local: tried
[warn] file:/Users/R2D2/Downloads/activator-dist-1.3.12/repository/org.apache.spark/spark-core_2.10/2.3.1/ivys/ivy.xml
[warn] ==== public: tried
[warn] http://repo1.maven.org/maven2/org/apache/spark/spark-core_2.10/2.3.1/spark-core_2.10-2.3.1.pom
[warn] ==== typesafe-releases: tried
[warn] https://repo.typesafe.com/typesafe/releases/org/apache/spark/spark-core_2.10/2.3.1/spark-core_2.10-2.3.1.pom
[warn] ==== typesafe-ivy-releasez: tried
[warn] https://repo.typesafe.com/typesafe/ivy-releases/org.apache.spark/spark-core_2.10/2.3.1/ivys/ivy.xml
[warn] ==== Typesafe Releases Repository: tried
[warn] http://repo.typesafe.com/typesafe/releases/org/apache/spark/spark-core_2.10/2.3.1/spark-core_2.10-2.3.1.pom
[info] Resolving org.jacoco#org.jacoco.agent;0.7.1.201405082137 ...
[error] impossible to get artifacts when data has not been loaded. IvyNode = org.slf4j#slf4j-api;1.6.6
java.lang.IllegalStateException: impossible to get artifacts when data has not been loaded. IvyNode = org.slf4j#slf4j-api;1.6.6
at org.apache.ivy.core.resolve.IvyNode.getArtifacts(IvyNode.java:809)
at org.apache.ivy.core.resolve.IvyNode.getSelectedArtifacts(IvyNode.java:786)
at org.apache.ivy.core.report.ResolveReport.setDependencies(ResolveReport.java:235)
at org.apache.ivy.core.resolve.ResolveEngine.resolve(ResolveEngine.java:235)
at org.apache.ivy.Ivy.resolve(Ivy.java:517)
at sbt.IvyActions$.sbt$IvyActions$$resolve(IvyActions.scala:211)
at sbt.IvyActions$$anonfun$update$1.apply(IvyActions.scala:126)
at sbt.IvyActions$$anonfun$update$1.apply(IvyActions.scala:125)
at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:116)
at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:116)
at sbt.IvySbt$$anonfun$withIvy$1.apply(Ivy.scala:104)
at sbt.IvySbt.sbt$IvySbt$$action$1(Ivy.scala:51)
at sbt.IvySbt$$anon$3.call(Ivy.scala:60)
at xsbt.boot.Locks$GlobalLock.withChannel$1(Locks.scala:93)
at xsbt.boot.Locks$GlobalLock.xsbt$boot$Locks$GlobalLock$$withChannelRetries$1(Locks.scala:78)
at xsbt.boot.Locks$GlobalLock$$anonfun$withFileLock$1.apply(Locks.scala:97)
at xsbt.boot.Using$.withResource(Using.scala:10)
at xsbt.boot.Using$.apply(Using.scala:9)
at xsbt.boot.Locks$GlobalLock.ignoringDeadlockAvoided(Locks.scala:58)
at xsbt.boot.Locks$GlobalLock.withLock(Locks.scala:48)
at xsbt.boot.Locks$.apply0(Locks.scala:31)
at xsbt.boot.Locks$.apply(Locks.scala:28)
at sbt.IvySbt.withDefaultLogger(Ivy.scala:60)
at sbt.IvySbt.withIvy(Ivy.scala:101)
at sbt.IvySbt.withIvy(Ivy.scala:97)
at sbt.IvySbt$Module.withModule(Ivy.scala:116)
at sbt.IvyActions$.update(IvyActions.scala:125)
at sbt.Classpaths$$anonfun$sbt$Classpaths$$work$1$1.apply(Defaults.scala:1170)
at sbt.Classpaths$$anonfun$sbt$Classpaths$$work$1$1.apply(Defaults.scala:1168)
at sbt.Classpaths$$anonfun$doWork$1$1$$anonfun$73.apply(Defaults.scala:1191)
at sbt.Classpaths$$anonfun$doWork$1$1$$anonfun$73.apply(Defaults.scala:1189)
at sbt.Tracked$$anonfun$lastOutput$1.apply(Tracked.scala:35)
at sbt.Classpaths$$anonfun$doWork$1$1.apply(Defaults.scala:1193)
at sbt.Classpaths$$anonfun$doWork$1$1.apply(Defaults.scala:1188)
at sbt.Tracked$$anonfun$inputChanged$1.apply(Tracked.scala:45)
at sbt.Classpaths$.cachedUpdate(Defaults.scala:1196)
at sbt.Classpaths$$anonfun$updateTask$1.apply(Defaults.scala:1161)
at sbt.Classpaths$$anonfun$updateTask$1.apply(Defaults.scala:1139)
at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:42)
at sbt.std.Transform$$anon$4.work(System.scala:64)
at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:237)
at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:237)
at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:18)
at sbt.Execute.work(Execute.scala:244)
at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:237)
at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:237)
at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:160)
at sbt.CompletionService$$anon$2.call(CompletionService.scala:30)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[error] (*:update) java.lang.IllegalStateException: impossible to get artifacts when data has not been loaded. IvyNode = org.slf4j#slf4j-api;1.6.6
[error] Total time: 24 s, completed Sep 10, 2018 12:03:30 PM
+ cd target/universal
./compile.sh: line 152: cd: target/universal: No such file or directory
++ /bin/ls '*.zip'
ls: *.zip: No such file or directory
+ ZIP_NAME=
+ unzip
+ rm
usage: rm [-f | -i] [-dPRrvW] file ...
unlink file
+ DIST_NAME=
+ chmod +x /bin/dr-elephant
chmod: /bin/dr-elephant: No such file or directory
+ sed -i.bak '/declare -r app_classpath/s/.$/:`hadoop classpath`:${ELEPHANT_CONF_DIR}"/' /bin/dr-elephant
sed: /bin/dr-elephant: No such file or directory
+ sudo cp /Users/R2D2/Desktop/dr-elephant/scripts/start.sh /bin/
Password:
cp: /bin/start.sh: Operation not permitted
+ sudo cp /Users/R2D2/Desktop/dr-elephant/scripts/stop.sh /bin/
cp: /bin/stop.sh: Operation not permitted
+ sudo cp -r /Users/R2D2/Desktop/dr-elephant/app-conf
usage: cp [-R [-H | -L | -P]] [-fi | -n] [-apvXc] source_file target_file
cp [-R [-H | -L | -P]] [-fi | -n] [-apvXc] source_file ... target_directory
+ sudo mkdir /scripts/
mkdir: /scripts/: File exists
+ sudo cp -r /Users/R2D2/Desktop/dr-elephant/scripts/pso /scripts/
+ zip -r .zip
zip error: Nothing to do! (.zip)
+ mv .zip /Users/R2D2/Desktop/dr-elephant/dist/
mv: rename .zip to /Users/R2D2/Desktop/dr-elephant/dist/.zip: No such file or directory
I'm a bit late to the party but this thread might be helpful in resolving the Ivy issue: https://github.com/sbt/sbt/issues/2015
And for point 2, the target/universal directory might not be created if the compilation is failing.
Upgrading to sbt 0.13.9 worked for me. But finally got compilation errors because some classes are removed in spark 2.3.0 while migration from spark 2.2.1
@ColinArmstrong were you successful in getting suggestions for spark 2.3 jobs ?