cloud-automation
cloud-automation copied to clipboard
changing the etl job to use spark as a sidecar image
Improvements
Spark wasn't getting rolled during "gen3 roll all". We decided it would be best for Spark to be a sidecar image instead of running it in a separate pod. This way spark will only run if an ETL job is triggered and it will always pull the image defined in the manifest.json file.