James Clarke

icon location London, UK icon location Senior Software/Data Engineer, Data Team, SwiftKey at Microsoft ─ Data Engineering, Python, Kotlin, Java, Spark, Databricks, Azure, AWS

Results 1 issues of James Clarke

Feature: Add extra spark-submit options

The following `spark-submit` options are really useful for tweaking the spark configuration/environment on a per-app/job basis: - `--packages` - `--exclude-packages` - `--repositories` - `--conf` - `--properties-file` The changes to the...