linkis
linkis copied to clipboard
[Question] How to set the running parameters of Spark
How to set the running parameters of Spark? such as
- spark.executor.cores,
- spark.executor.memory
-
if you run by linkis-cli such as
sh ./bin/linkis-cli -engineType spark-2.4.3 -codeType sql -confMap wds.linkis.yarnqueue=q02 -confMap spark.executor.instances=3 -code "select count(*) from testdb.test;" -submitUser hadoop -proxyUser hadoop
see detail https://linkis.apache.org/zh-CN/docs/1.1.3/user_guide/linkiscli-manual/ -
system level add new system level configuration for creator in linkis web console
-
set for single task in dss (like eg 1)