docker-containers icon indicating copy to clipboard operation
docker-containers copied to clipboard

Add dnsutils package to mesosphere/spark for service discovery

Open mcapuccini opened this issue 8 years ago • 0 comments

It would be very nice to have dnsutils installed in the mesosphere/spark container. This would make possible to do service discovery using mesos-dns or consul. This would enable the users to submit this kind of commands through Chronos:

MASTER_PORT=$(dig +short spark-dispatcher.service.consul SRV | awk '{print $3}' | sort | head -1) && \
bin/spark-submit --class org.apache.spark.examples.SparkPi \ 
--master mesos://spark-dispatcher.service.consul:$MASTER_PORT \
--deploy-mode cluster \
/mnt/container-volumes/spark-1.6.0-bin-hadoop2.4/lib/spark-examples-1.6.0-hadoop2.4.0.jar \
1000

mcapuccini avatar Mar 11 '16 10:03 mcapuccini