Error when setting timezone for jvm in spark.executor.extraJavaOptions and spark.driver.extraJavaOptions

[AadhiKat]

Used Spark version -> 2.4.3

Used Spark Job Server version -> 0.9.0

Deployed mode -> Cluster Mode

local.conf

spark {
  master = "spark://xxx.xx.xx.xx:xxxx,yyy.yy.yy.yy:yyyy"
  submit.deployMode = "cluster"
  job-number-cpus = 4

  jobserver {
    port = 9090
    context-per-jvm = true
    ignore-akka-hostname=false
    jobdao = spark.jobserver.io.JobSqlDAO

    filedao {
      rootdir = /home/sas/zdpas/job-server/filedao/data
    }

    datadao {
      rootdir = /home/sas/zdpas/job-server/upload
    }

    sqldao {
      slick-driver = slick.driver.PostgresDriver
      jdbc-driver = org.postgresql.Driver
      rootdir = /home/sas/zdpas/job-server/sqldao/data
      jdbc {
        url = "jdbc:postgresql://zzz.zz.zz.zz:zzzz/spark_jobserver"
        user = "jobserver"
        password = ""
      }
      dbcp {
        enabled = false
        maxactive = 20
        maxidle = 10
        initialsize = 10
      }
    }
    result-chunk-size = 1m
  }

  context-settings {
    passthrough {
		spark.executor.extraJavaOptions = "-Duser.timezone=Asia/Kolkata"
		spark.driver.extraJavaOptions =  "-Duser.timezone=Asia/Kolkata"
      #es.nodes = "192.1.1.1"
    }
  }
  home = "/home/sas/zdpas/spark"
}

akka {
  remote.netty.tcp {
	hostname="zzz.zz.zz.zz"

  # maximum-frame-size = 10 MiB
  }
}

flyway.locations="db/postgresql/migration"

shiro{
  authentication = on
  config.path = /home/sas/zdpas/job-server/shiro.ini
  authentication-timeout = 10s
}


spray.can.server {
	idle-timeout = 120s
	request-timeout = 100s
	parsing {
		max-content-length = 30m
		#max-uri-length = 8k
	}
}

It seems to be listed in the spark properties , but it dosent change the user.timezone in the System properties , whereas when I do it in my local spark I am able to replace it in the user.timezone property , in the spark-defaults.conf file.

What should be done so that the user.timezone is changed in the System Properties.

[thoHeinze]

@AadhiKat I think the way you tried it seems to be correct, I'm not 100% sure why it is not working. The jobserver tries to set these properties in the SparkConf object during runtime. I'm not sure if this works for such parameters.

[whaitukay]

I've had a similar problem setting spark.jars.packages a while back. The root issue was that these configurations are being added to the SparkConf (after the JVMs have started up), which is a little bit too late. The solution I found was to add the required configurations to the spark.context-settings.launcher config when creating a context. Everything then gets added to the spark-submit command, which might be what you are looking for.

Maybe this will be of some help?

spark {
  context-settings {
    launcher {
      spark.executor.extraJavaOptions = "-Duser.timezone=Asia/Kolkata"
      spark.driver.extraJavaOptions =  "-Duser.timezone=Asia/Kolkata"
    }
  }
}