sparklens icon indicating copy to clipboard operation
sparklens copied to clipboard

Not able to run spark lens on spark history file

Open Gaurang033 opened this issue 4 years ago • 1 comments

hi Guys,

I am trying to get spark lens report from my history file. However it's not able to identify the arguments.

I am getting following error

[sa_awsprd_nztoaws@ip-10-160-1-109 gshah]$ spark-submit --jars sparklens-0.3.2-s_2.11.jar  --class com.qubole.sparklens.app.ReporterApp qubole-dummy-arg  file:////home/gshah/application_1589573238486_1924 source=file
20/09/03 14:57:22 WARN DependencyUtils: Local jar /home/gshah/qubole-dummy-arg does not exist, skipping.
#
# java.lang.OutOfMemoryError: Java heap space
# -XX:OnOutOfMemoryError="kill -9 %p"
#   Executing /bin/sh -c "kill -9 3788"...
Killed

Gaurang033 avatar Sep 03 '20 15:09 Gaurang033

@Gaurang033 looks like you are running our of driver heap. Try increasing driver memory: spark-submit --driver-memory 4g --jars sparklens-0.3.2-s_2.11.jar --class com.qubole.sparklens.app.ReporterApp qubole-dummy-arg file:////home/gshah/application_1589573238486_1924 source=file

iamrohit avatar Sep 06 '20 06:09 iamrohit