spark-operator
spark-operator copied to clipboard
sparkConf creates incorrect path from `spark.jars.packages` and `spark.jars.repositories`
Using spark-operator:v1beta2-1.2.0-3.0.0, and creating a SparkApplication as follows:
apiVersion: "sparkoperator.k8s.io/v1beta2"
kind: SparkApplication
metadata:
name: pyspark-pi
namespace: spark-operator
spec:
sparkConf:
spark.jars.packages: "org.apache.hadoop:hadoop-aws:2.7.6,com.amazonaws:aws-java-sdk:aws-java-sdk-1.7.6"
spark.jars.repositories: <redacted>
...
The url that is created is incorrect:
<redacted>/com/amazonaws/aws-java-sdk/aws-java-sdk-1.7.6/aws-java-sdk-aws-java-sdk-1.7.6.jar
Note the repeated aws-java-sdk-aws-java-sdk
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
This issue has been automatically closed because it has not had recent activity. Please comment "/reopen" to reopen it.