error installing on macos 10.14.x

[staminna]

In order of execution, the instalation fails on latest macos 10.14.x with the following issues

Step 8/11 : ADD target/scala-2.11/search-flight-spark-streaming-assembly-0.1.0.jar $SPARK_HOME/libs/search-flight-streaming.jar
ERROR: Service 'streaming-spark-dev' failed to build: ADD failed: stat /var/lib/docker/tmp/docker-builder288103139/target/scala-2.11/search-flight-spark-streaming-assembly-0.1.0.jar: no such file or directory

Step 3/6 : ADD target/search-flight-hadoop-batch-0.1.0-jar-with-dependencies.jar $HADOOP_HOME/libs/search-flight-batch.jar
ERROR: Service 'batch-hadoop-dev' failed to build: ADD failed: stat /var/lib/docker/tmp/docker-builder770657281/target/search-flight-hadoop-batch-0.1.0-jar-with-dependencies.jar: no such file or directory

Step 4/6 : ADD target/search-flight-storm-streaming-0.1.0-jar-with-dependencies.jar $STORM_HOME/libs/search-flight-streaming.jar
ERROR: Service 'streaming-storm-dev' failed to build: ADD failed: stat /var/lib/docker/tmp/docker-builder383930762/target/search-flight-storm-streaming-0.1.0-jar-with-dependencies.jar: no such file or directory

[Chabane]

hello, thanks for your contribution I will check

[staminna]

Hi, @Chabane Thanks for quick reply. I hope you can reproduce

[Chabane]

I have to investigate for 0.14.x

It works with OSX 10.13.6

$ git clone https://github.com/Chabane/bigdata-playground
$ cd bigdata-playground
$ cd streaming/spark && sbt clean assembly && cd ../..
$ cd docker
$ docker-compose -f dev/streaming-spark.yml up -d
Building streaming-spark-dev
Step 1/11 : FROM openjdk:8-jre
8-jre: Pulling from library/openjdk
741437d97401: Pull complete
34d8874714d7: Pull complete
0a108aa26679: Pull complete
eea0f08c7492: Pull complete
ca74b3d90c15: Pull complete
f11a643c4a85: Pull complete
05eafed73b65: Pull complete
Digest: sha256:2216ccda45993afaa42b3b269aa1ae8832cb2fabbf8207f63d528fdc7c8a087b
Status: Downloaded newer image for openjdk:8-jre
 ---> ed287c436e66
Step 2/11 : VOLUME /tmp
 ---> Running in 88436c00ffe6
Removing intermediate container 88436c00ffe6
 ---> 8b4ae4f0795f
Step 3/11 : ENV SPARK_VERSION spark-2.4.0-bin-hadoop2.7
 ---> Running in 18f69569d784
Removing intermediate container 18f69569d784
 ---> 43ec961a3688
Step 4/11 : ENV SPARK_HOME /usr/local/spark
 ---> Running in 3185f165f176
Removing intermediate container 3185f165f176
 ---> d45e0984fe65
Step 5/11 : RUN curl https://archive.apache.org/dist/spark/spark-2.4.0/$SPARK_VERSION.tgz -o $SPARK_VERSION.tgz;     tar xzf $SPARK_VERSION.tgz -C /usr/local/;
 ---> Running in f8d5c2aeea09
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  217M  100  217M    0     0   9.8M      0  0:00:22  0:00:22 --:--:-- 9401k
Removing intermediate container f8d5c2aeea09
 ---> 904bc6c8ec6b
Step 6/11 : RUN cd /usr/local && ln -s $SPARK_VERSION spark
 ---> Running in 1afb5821bc35
Removing intermediate container 1afb5821bc35
 ---> d4d112977ae5
Step 7/11 : ADD hbase-site.xml $SPARK_HOME/conf
 ---> 855d1f6f0aaf
Step 8/11 : ADD target/scala-2.11/search-flight-spark-streaming-assembly-0.1.0.jar $SPARK_HOME/libs/search-flight-streaming.jar
 ---> 7912943b00bd
Step 9/11 : RUN sh -c 'touch $SPARK_HOME/libs/search-flight-streaming.jar'
 ---> Running in 596e323e7cd5
Removing intermediate container 596e323e7cd5
 ---> 42bcd548a74e
Step 10/11 : WORKDIR $SPARK_HOME
 ---> Running in 5e91d58ce3fa
Removing intermediate container 5e91d58ce3fa
 ---> 09217c527d4e
Step 11/11 : CMD ["bin/spark-submit", "--class", "com.mitosis.Main", "--master", "local[*]", "--files", "/usr/local/spark/conf/hbase-site.xml", "./libs/search-flight-streaming.jar"]
 ---> Running in 05cea3bcbca9
Removing intermediate container 05cea3bcbca9
 ---> d16a3d45765a

Successfully built d16a3d45765a
Successfully tagged dev_streaming-spark-dev:latest
WARNING: Image for service streaming-spark-dev was built because it did not already exist. To rebuild this image you must use `docker-compose build` or `docker-compose up --build`.
Creating streaming_spark_dev ... done