Feature: Add extra spark-submit options

Open jamesclarke opened this issue 6 years ago • 0 comments

The following spark-submit options are really useful for tweaking the spark configuration/environment on a per-app/job basis:

--packages
--exclude-packages
--repositories
--conf
--properties-file

The changes to the code that enable them are simply additional arguments to the existing API/CLI and just pass them straight through to the spark submit command generator. To make things consistent with the spark-submit CLI, the corresponding arguments in AZTK maintain the same names.

Feb 13 '19 16:02 jamesclarke