sparklens
sparklens copied to clipboard
Not able to run spark lens on spark history file
hi Guys,
I am trying to get spark lens report from my history file. However it's not able to identify the arguments.
I am getting following error
[sa_awsprd_nztoaws@ip-10-160-1-109 gshah]$ spark-submit --jars sparklens-0.3.2-s_2.11.jar --class com.qubole.sparklens.app.ReporterApp qubole-dummy-arg file:////home/gshah/application_1589573238486_1924 source=file
20/09/03 14:57:22 WARN DependencyUtils: Local jar /home/gshah/qubole-dummy-arg does not exist, skipping.
#
# java.lang.OutOfMemoryError: Java heap space
# -XX:OnOutOfMemoryError="kill -9 %p"
# Executing /bin/sh -c "kill -9 3788"...
Killed
@Gaurang033 looks like you are running our of driver heap. Try increasing driver memory:
spark-submit --driver-memory 4g --jars sparklens-0.3.2-s_2.11.jar --class com.qubole.sparklens.app.ReporterApp qubole-dummy-arg file:////home/gshah/application_1589573238486_1924 source=file