adam icon indicating copy to clipboard operation
adam copied to clipboard

Cannot run pyadam adam-distribution-spark2_2.11-0.24.0 binary distribution

Open mtdeguzis opened this issue 6 years ago • 4 comments

bash -x /usr/local/adam/bin/find-adam-egg.sh
+ set -e
+++ readlink -f /usr/local/adam/bin/find-adam-egg.sh
++ dirname /usr/local/adam/bin/find-adam-egg.sh
+ SOURCE_DIR=/usr/local/adam/bin
+ . /usr/local/adam/bin/find-adam-home
++ set -e
++++ dirname /usr/local/adam/bin/find-adam-egg.sh
+++ cd /usr/local/adam/bin
+++ pwd
++ FIND_ADAM_HOME_PYTHON_SCRIPT=/usr/local/adam/bin/find_adam_home.py
++ '[' '!' -z '' ']'
++ '[' '!' -f /usr/local/adam/bin/find_adam_home.py ']'
++++ dirname /usr/local/adam/bin/find-adam-egg.sh
+++ cd /usr/local/adam/bin/..
+++ pwd
++ export ADAM_HOME=/usr/local/adam
++ ADAM_HOME=/usr/local/adam
+ '[' -d /usr/local/adam/repo ']'
+ DIST_DIR=/usr/local/adam/repo
++ ls -1 /usr/local/adam/repo
++ grep '^bdgenomics\.adam[0-9A-Za-z\.\_\-]*.egg$'
++ true
+ DIST_EGG=
++ echo
++ wc -l
+ num_egg=1
+ '[' 1 -eq 0 ']'
+ '[' 1 -gt 1 ']'
+ echo /usr/local/adam/repo/
/usr/local/adam/repo/

The script does not report the egg dir is missing, even though DIST_EGG is blank.

[mtdeguzis@mtdeguzis-hdpcli: ~]$ ls -la /usr/local/adam/
total 212
drwxr-xr-x.  6 mtdeguzis mtdeguzis    139 Aug 31 15:08 .
drwxr-xr-x. 14 root      root         162 Aug 31 15:03 ..
drwxr-xr-x.  2 mtdeguzis mtdeguzis    166 Aug 31 15:31 bin
-rw-r--r--.  1 mtdeguzis mtdeguzis 173242 Jan  4  2018 CHANGES.md
-rw-r--r--.  1 mtdeguzis mtdeguzis   4514 Jan  4  2018 CONTRIBUTING.md
drwxr-xr-x.  3 mtdeguzis mtdeguzis    279 Jan  4  2018 javadocs
-rw-r--r--.  1 mtdeguzis mtdeguzis  16714 Jan  4  2018 LICENSE.txt
-rw-r--r--.  1 mtdeguzis mtdeguzis   9345 Jan  4  2018 README.md
drwxr-xr-x.  2 mtdeguzis mtdeguzis    154 Aug 31 15:06 repo
drwxr-xr-x.  5 mtdeguzis mtdeguzis    123 Jan  4  2018 scaladocs
[mtdeguzis@mtdeguzis-hdpcli: ~]$ vim /usr/local/adam/bin/find-adam-egg.sh
[mtdeguzis@mtdeguzis-hdpcli: ~]$ ls -la /usr/local/adam/repo
total 53088
drwxr-xr-x. 2 mtdeguzis mtdeguzis      154 Aug 31 15:06 .
drwxr-xr-x. 6 mtdeguzis mtdeguzis      139 Aug 31 15:08 ..
-rw-r--r--. 1 mtdeguzis mtdeguzis 54005157 Jan  4  2018 adam-assembly-spark2_2.11-0.23.0.jar
-rw-r--r--. 1 mtdeguzis mtdeguzis   350930 Jan  4  2018 adam-assembly-spark2_2.11-0.23.0-javadoc.jar
-rw-r--r--. 1 mtdeguzis mtdeguzis     3469 Jan  4  2018 adam-assembly-spark2_2.11-0.23.0-sources.jar

mtdeguzis avatar Aug 31 '18 15:08 mtdeguzis

Error:

: org.apache.spark.SparkException: Added file file:/usr/local/adam/repo is a directory and recursive is not turned on.

Cannot locate egg in binary dist download

mtdeguzis avatar Aug 31 '18 15:08 mtdeguzis

Issue was fixed after this dist release

mtdeguzis avatar Aug 31 '18 15:08 mtdeguzis

get the right error with the 24.0 tar.gz distribution

 [mtdeguzis@mtdeguzis-hdpcli: ~]$ pyadam
Failed to find ADAM egg in /usr/local/adam/repo.
You need to build ADAM before running this program.

repo folder still only has 3 jar files in it

  • Used https://repo1.maven.org/maven2/org/bdgenomics/adam/adam-distribution-spark2_2.11/0.24.0/
  • filename: https://repo1.maven.org/maven2/org/bdgenomics/adam/adam-distribution-spark2_2.11/0.24.0/adam-distribution-spark2_2.11-0.24.0-bin.tar.gz

mtdeguzis avatar Aug 31 '18 16:08 mtdeguzis

I believe this may still be an issue, even with the 0.27.0 release cut today.

heuermh avatar May 23 '19 04:05 heuermh