spark-operator icon indicating copy to clipboard operation
spark-operator copied to clipboard

How to add volumes or ephemeral storage to spark application configuration used by Spark Operator

Open krisshub opened this issue 4 years ago • 0 comments

Description:

Is there an option to add volume or increase ephemeral storage for the spark application

Steps to reproduce:

apiVersion: radanalytics.io/v1
kind: SparkApplication
metadata:
  name: xxx
  namespace: default
spec:
  type: Scala
  mode: cluster
  image: "xxx"
  imagePullPolicy: Always
  mainClass: xxx
  mainApplicationFile: "xxx"
  arguments: "xxx"
  restartPolicy: Never
  sparkVersion: "3.0.0"
  driver:
    cores: 1
    coreLimit: "1000m"
    memory: "10240m"
    labels:
      version: 3.0.0
  executor:
    cores: 2
    coreLimit: "2000m"
    memory: "12288m"
    instances: 44
    labels:
      version: 3.0.0

krisshub avatar Jun 12 '21 09:06 krisshub