arena icon indicating copy to clipboard operation
arena copied to clipboard

pyspark support in sparkjob

Open Alienfeel opened this issue 3 years ago • 0 comments

Lang: Scala

Image: registry.aliyuncs.com/acs/spark:v2.4.0

MainClass: org.apache.spark.examples.SparkPi

Jar: local:///opt/spark/examples/jars/spark-examples_2.11-2.4.0.jar

Driver:
  CPURequest: 1
  MemoryRequest: "1g"

Executor:
  CPURequest: 1
  MemoryRequest: "1g"
  Replicas: 1
  

only support Scala task,however, pyspark is quite popular in data sciencie task

when will pyspark task available in arena ?

Alienfeel avatar Jul 15 '21 03:07 Alienfeel