adam
adam copied to clipboard
Cannot run pyadam adam-distribution-spark2_2.11-0.24.0 binary distribution
bash -x /usr/local/adam/bin/find-adam-egg.sh
+ set -e
+++ readlink -f /usr/local/adam/bin/find-adam-egg.sh
++ dirname /usr/local/adam/bin/find-adam-egg.sh
+ SOURCE_DIR=/usr/local/adam/bin
+ . /usr/local/adam/bin/find-adam-home
++ set -e
++++ dirname /usr/local/adam/bin/find-adam-egg.sh
+++ cd /usr/local/adam/bin
+++ pwd
++ FIND_ADAM_HOME_PYTHON_SCRIPT=/usr/local/adam/bin/find_adam_home.py
++ '[' '!' -z '' ']'
++ '[' '!' -f /usr/local/adam/bin/find_adam_home.py ']'
++++ dirname /usr/local/adam/bin/find-adam-egg.sh
+++ cd /usr/local/adam/bin/..
+++ pwd
++ export ADAM_HOME=/usr/local/adam
++ ADAM_HOME=/usr/local/adam
+ '[' -d /usr/local/adam/repo ']'
+ DIST_DIR=/usr/local/adam/repo
++ ls -1 /usr/local/adam/repo
++ grep '^bdgenomics\.adam[0-9A-Za-z\.\_\-]*.egg$'
++ true
+ DIST_EGG=
++ echo
++ wc -l
+ num_egg=1
+ '[' 1 -eq 0 ']'
+ '[' 1 -gt 1 ']'
+ echo /usr/local/adam/repo/
/usr/local/adam/repo/
The script does not report the egg dir is missing, even though DIST_EGG is blank.
[mtdeguzis@mtdeguzis-hdpcli: ~]$ ls -la /usr/local/adam/
total 212
drwxr-xr-x. 6 mtdeguzis mtdeguzis 139 Aug 31 15:08 .
drwxr-xr-x. 14 root root 162 Aug 31 15:03 ..
drwxr-xr-x. 2 mtdeguzis mtdeguzis 166 Aug 31 15:31 bin
-rw-r--r--. 1 mtdeguzis mtdeguzis 173242 Jan 4 2018 CHANGES.md
-rw-r--r--. 1 mtdeguzis mtdeguzis 4514 Jan 4 2018 CONTRIBUTING.md
drwxr-xr-x. 3 mtdeguzis mtdeguzis 279 Jan 4 2018 javadocs
-rw-r--r--. 1 mtdeguzis mtdeguzis 16714 Jan 4 2018 LICENSE.txt
-rw-r--r--. 1 mtdeguzis mtdeguzis 9345 Jan 4 2018 README.md
drwxr-xr-x. 2 mtdeguzis mtdeguzis 154 Aug 31 15:06 repo
drwxr-xr-x. 5 mtdeguzis mtdeguzis 123 Jan 4 2018 scaladocs
[mtdeguzis@mtdeguzis-hdpcli: ~]$ vim /usr/local/adam/bin/find-adam-egg.sh
[mtdeguzis@mtdeguzis-hdpcli: ~]$ ls -la /usr/local/adam/repo
total 53088
drwxr-xr-x. 2 mtdeguzis mtdeguzis 154 Aug 31 15:06 .
drwxr-xr-x. 6 mtdeguzis mtdeguzis 139 Aug 31 15:08 ..
-rw-r--r--. 1 mtdeguzis mtdeguzis 54005157 Jan 4 2018 adam-assembly-spark2_2.11-0.23.0.jar
-rw-r--r--. 1 mtdeguzis mtdeguzis 350930 Jan 4 2018 adam-assembly-spark2_2.11-0.23.0-javadoc.jar
-rw-r--r--. 1 mtdeguzis mtdeguzis 3469 Jan 4 2018 adam-assembly-spark2_2.11-0.23.0-sources.jar
Error:
: org.apache.spark.SparkException: Added file file:/usr/local/adam/repo is a directory and recursive is not turned on.
Cannot locate egg in binary dist download
Issue was fixed after this dist release
get the right error with the 24.0 tar.gz distribution
[mtdeguzis@mtdeguzis-hdpcli: ~]$ pyadam
Failed to find ADAM egg in /usr/local/adam/repo.
You need to build ADAM before running this program.
repo folder still only has 3 jar files in it
- Used https://repo1.maven.org/maven2/org/bdgenomics/adam/adam-distribution-spark2_2.11/0.24.0/
- filename: https://repo1.maven.org/maven2/org/bdgenomics/adam/adam-distribution-spark2_2.11/0.24.0/adam-distribution-spark2_2.11-0.24.0-bin.tar.gz
I believe this may still be an issue, even with the 0.27.0 release cut today.