Integration with Spark Magic.

[as-sher]

Hi,

I am using spark magic(https://github.com/jupyter-incubator/sparkmagic) which provides me with kernels to run my commands on remote clusters. Is there a way to integrate this spark monitor to it?

[itsjafer]

I'll have to set up spark magic to try and test it out which will probably take a while since it's not really high priority for me.

Could you try following the installation instructions with the exception that, when creating a new SparkSession using sparkmagics, you add the following configs:

'spark.extraListeners': 'sparkmonitor.listener.JupyterSparkMonitorListener'
'spark.driver.extraClassPath': 'path/to/your/python/packages/sparkmonitor/listener.jar'

[utkarshgupta137]

Hey, I tried this and it is not working. We use spark magic to connect to spark in cluster mode. I guess the problem will be that the extension will have to connect to the spark on a remote cluster. Let me know if I can help debugging the issue.